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Abstract 

Sifvctnl  investigations  liave  fonini  that  students  learn  more  when  they  exi>lain 
examples  to  themselves  wliile  studyitig  tliein.  Moreover,  they  refer  less  often  to 
the  examples  while  solvitig  problems,  and  they  read  less  of  the  example  each 
time  they  refer  to  it.  1  hese  findings,  collectively  called  the  self-explanation 
efiert,  have  been  reproduced  by  our  cognitive  sinnilation  program.  Cascade. 
Moreover,  when  Cascade  is  forced  to  explain  exactly  the  parts  of  the  examples 
that  a  subject  explains,  then  it  predicts  most  (60  to  90%)  of  the  behavior 
that  the  snbj<>rt  exhiliits  during  sitbsequent  problem  solving.  Cascade  has  two 
kinds  fif  learniitg  It  learns  new  rules  of  |>hysics  (the  task  domain  used  in  the 
hntnan  data  modeled)  by  resolving  impa.sses  with  reasoning  based  on  overly- 
general.  non-domaiti  knowledge.  It  acquires  procedural  competence  by  storing 
its  derivat  iotis  of  problem  sohttiotis  atid  using  them  as  analogs  to  guide  its  search 
for  solutions  to  ttovel  problems. 


THE  TWO  MAJOR  OBJECTIVES  OF  THE 
CASCADE  PROJECT 

.As  Tom  Dietterich  pointed  out  in  the  keynote  address  of  the  1990 
Machine  Learninp;  Conference,  one  of  the  biggest  challenges  in  machine 
learning  is  to  get  inacliines  to  learn  from  ordinary  instructional  material, 
such  as  that  used  to  train  scientists,  engineers  and  technicians.  Not 

’This  research  was  supported  bv  the  Cognitive  Science  division  of  the  Office  of 
Naval  Research  under  contract  N00ni4-8S-K-00S6  and  the  Information  Sciences  divi¬ 
sion  of  the  Office  of  Nava!  Research  under  contract  N()n014-86-K-0678. 


only  is  this  an  exciting  intellectual  challenge,  but  it  might  help  alleviate 
the  noinrions  pif)l)leni  of  getting  expertise  out  of  the  culture  of  experts 
and  into  an  operable  form.  The  expert  systems  community  recently 
realized  that  not  all  experts  are  good  at  explicating  and  explaining  their 
knowledge,  but  instructors  vary  in  quality  too.  so  a  common  practice 
nowadays  is  to  ac{|uire  knowledge  for  an  expert  system  from  an  expert 
who  is  also  a  good  instructor.  Often,  there  are  textbooks  written  by 
very  good  instructors.  Utilizing  this  material  requires  having  the  kind 
of  system  tnat  Dietterich  envisioned. 

The  j)rogram  described  here.  Cascade,  is  a  direct  response  to  Di¬ 
et  lerich's  challenge,  for  it  can  learn  how  to  solve  Newtonian  mechanics 
problems  from  the  same  materials  that  undergraduates  learn  from.  How¬ 
ever.  it  is  oidy  a  partial  solution  to  the  problem,  because  Cascade  cannot 
read.  The  information  in  the  prose  parts  of  the  textbook  is  given  to  it 
in  a  predigested  form.  .As  will  be  demonstrated  later,  this  information 
is  not  as  helpful  in  solving  problems  as  one  might  think.  People  and 
Casc  ade  acquire  much  of  their  problem  solving  skill  by  solving  problems 
and  by  studying  the  textbook's  worked  example  problems. 

The  second  ol)jective  of  the  research  presented  here  is  to  integrate 
and  dec'pen  the  theory  of  skill  acquisition.  As  theories  go.  the  theory  of 
cognitive  skill  aceptisition  is  in  its  infancy.  Theories  range  all  the  way 
frotn  the  highly  integrated,  nomological  theories  of  certain  natural  sci¬ 
ences  to  loose  collections  of  ideas  which  can  be  woven  together  toexplain 
pitenomena.  The  current  theory  of  cognitive  skill  acquisition  is  in  the 
collect ion-of-ideas  stage.  Given  almost  any  behavior,  a  cognitive  scien¬ 
tist  can  often  string  together  ideas  from  psychology  and  .41  that  will  offer 
at  least  a  plausible  explanation  of  the  phenomenon.  This  is  certainly  an 
advance  over  the  state  of  the  art  25  years  ago.  However,  the  theory  is 
not  as  integrated  as  it  could  l)e.  For  instance,  no  one  has  built  a  com¬ 
putational  model  of  skill  acquisition  that  starts  as  a  novice  and  slowly 
becomes  an  expert  while  being  trained  on  the  same  material  as  human 
students.  Several  models  of  pieces  of  this  process  have  been  built,  includ¬ 
ing  Sierra  (\'anl.ehn.  1990).  Pups  (.Anderson  A'  Thompson.  I9f^9)  and  X 
(Pirolli.  19S7).  The  reason  that  cognitive  science  has  no  simulated  stu¬ 
dents  is  not  just  because  it  is  technically  difficult,  but  because  we  do  not 
know  w'hich  of  the  many  ideas  floating  around  should  be  woven  together. 
Moving  the  theory  of  cognitive  skill  acquisition  out  of  the  coUection-of- 
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idrns  stage  aiul  into  a  stage  of  integrated  student  simulations  will  re- 
(|Miie  deep  tlioiiglit  ami  significant  new  empirical  work.  Development 
of  student  simulations  should  go  hand-in-hand  with  these  empirical  ad¬ 
vances.  because  such  simulations  are  the  only  way  to  demonstrate  the 
computational  coherence  and  empirical  coverage  of  an  integrated  theory 
of  cognitive  skill  accpiisition.  Cascade  is  intended  to  be  a  step  further  in 
that  it  incorporates  new  empirical  evidence  from  a  study  by  Chi.  Bas- 
sok.  Lewis.  Heimann  and  Glaser  (1989).  However.  Cascade  is  far  from  a 
complete  simulation,  because  some  important  cognitive  processes,  such 
as  reading,  have  been  deliberately  omitted  from  the  model. 

Interesting  new  educational  technology  may  result  from  developing 
the  simulated  students  that  are  required  of  an  integrated  theory  of  cog¬ 
nitive  skill  accpiisition.  For  instance,  a  simulated  student  might  be  a 
valuable  tool  for  training  teachers.  Simulators  have  been  successful  ad¬ 
juncts  in  training  other  skills,  ranging  from  flying  airplanes  to  trading 
stocks.  It  may  be  a  worthwhile  investment  to  use  simulators  to  train 
teailiers  in  the  skills  of  selecting  material  to  teach,  organizing  it.  e.\- 
plaining  it.  detecting  student  misconceptions  and  remediating  them.  In 
addition  to  teacher  training,  there  are  other  potential  applications  for 
siinnlale<l  sttidenis  as  well  (VanLehu.  1991b). 

The  lack  of  an  integrated  theory  prevents  development  of  many  ap- 
pli(alions.  not  just  educational  ones.  The  problem  is  that  a  theory  that 
is  in  the  collect ion-of-ideas  stage  often  provides  multiple  or  vague  e,\i)la- 
nations  of  |)henomena.  which  means  that  it  can  make  only  ambiguous 
or  vague  predictions  at  best.  Vet  many  applications,  such  as  simulation. 
re<|uire  the  theory  to  make  unambiguous,  precise  predictions.  Until  our 
understanding  of  cognitive  skill  acquisition  is  good  enough  that  we  can 
make  such  predictions,  many  applications  are  beyond  our  reach. 

This  rha|)ter  is  intended  as  a  summary  of  the  results  so  far  from  the 
Cascade  project.  The  project  has  gone  through  three  major  phases.  In 
the  first  phase,  the  program  was  developed  and  shown  capable  of  learn¬ 
ing  .Newtonian  mechanics  correctly  (VanLehn  iC  .Jones,  in  press).  In  the 
second  pliase.  the  major  findings  from  the  Chi  et  al.  study  were  simu¬ 
lated  (\anLehn.  .Jones  k  Chi.  in  press).  In  the  third  phase,  protocols  of 
each  of  the  9  subjects  in  the  Chi  sttidy  were  simulated  individually.  The 
third  phase  is  ongoing,  so  we  can  present  only  some  of  the  planned  analy¬ 
ses,  III  particular,  we  evaluate  the  overall  fit  of  Cascade  to  the  protocols. 


wilitli  is  important  for  seeing  liow  well  Cascade  functions  as  a  siinnlated 
sitidf’iM  and  ns  a  knowledge  ar(|nisi1ion  system  that  would  satisfy  Di¬ 
et  terich's  challenge.  This  chapter  follows  the  historical  development  hy 
first  describing  the  Chi  et  al.  (1989)  study,  then  describing  Cascade, 
then  d<‘scribing  how  Cascade  accounts  for  the  Chi  et  al.  findings,  then 
describing  how  it  simulates  individual  subjects. 

THE  SELF-EXPLANATION  EFFECT 

One  of  the  major  open  i.ssues  in  cognitive  skill  acquisition  is  under¬ 
standing  what  happens  when  people  study  e.xamples.  (An  e.xainple  is  a 
problem  together  witli  a  solution  that  is  printed  or  demonstrated  for  the 
student.)  Much  research  has  shown  that  when  people  are  given  instruc¬ 
tion  consisting  of  theory,  e.xamples  and  explanations,  they  rely  heavily 
on  the  e.xamples  (e.g..  Anderson.  Farrell  k  Saiirers.  1984:  Sweller  k 
Cooper.  198.j).  In  some  cases  they  seem  to  ignore  the  theory  and  expla¬ 
nations.  and  in  other  cases  their  learning  is  actually  retarded  by  them 
(e.g..  LeFevre  A'  Di.xon.  1986;  Charney.  Reder.  A:  Kusbit.  1990;  Waid  k 
Sweller.  1990).  Because  exami)les  seem  to  do  much  more  of  the  teaching 
than  was  previously  thought,  it  is  important  to  understaml  how  they 
work. 

Chi  et  al.  (  19S9)  took  a  direct  approach  to  understanding  how  stu¬ 
dents  study  examples.  They  collected  protocols  as  subjects  studied  ex- 
ami>les  in  classical  particle  dynamics,  the  first  topic  in  a  typical  first-year 
college  physics  course.  .Nine  subjects  studied  the  first  three  chapters  of  a 
college  textbook,  then  read  the  prose  part  of  a  chapter  on  Newton’s  laws. 
They  took  a  test  on  their  understanding  of  the  chapter,  then  studied  3 
exatnples  and  solved  2.')  problems.  Protocols  were  taken  as  they  studied 
the  exam|)les  and  solved  the  problems.  On  the  basis  of  the  scores  on 
proi)leni  solving,  the  subjects  were  divided  into  two  groups.  The  4  stu¬ 
dents  wit  h  the  highest  scores  were  called  the  Good  solvers:  the  4  students 
with  the  lowest  scores  were  called  Poor  solvers,  and  one  student  was  not 
analyzed  (see  t'hi  and  V'anLehn.  1991.  for  a  discussion  of  the  subjects' 
backgrounds  and  the  median-split  procedure).  Since  the  students  in  both 
groups  scored  the  same  on  pre-tests,  the  Good  solvers  seemed  to  have 
learned  more  during  the  experiment.  Using  protocol  analysis.  Chi  et  al. 
attempted  to  find  out  how  the  Good  solvers  managed  to  learn  more  than 
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the  Poor  solvers  from  the  same  material.  They  found  four  differences; 

1.  The  CJood  solvers  uttered  more  self-explanations  as  they  studied 
examples,  whereas  the  Poor  solvers’  comments  were  mostly  jrara- 
phrases  of  the  examples'  statements. 

2.  .All  students  commented  frequently  on  whether  they  understood 
what  they  had  just  read.  The  Good  solvers  tended  to  say  that 
they  did  not  understand  what  they  had  just  read,  whereas  the 
Poor  solvers  tended  to  say  that  they  did  understand.  However,  the 
Poor  solvers'  scores  show  that  they  understood  less  than  the  Good 
solvers.  This  indicates  that  the  Poor  solvers'  seif-monitoring  was 
less  accurate  than  the  Good  solvers'. 

d.  During  prohlem  solving,  the  Poor  solvers  tended  to  refer  back  to 
the  examples  more  often  than  the  Good  solvers. 

I.  When  the  Good  solvers  referred  to  the  examples,  they  read  fewer 
lines  than  the  Poor  solvers.  The  Poor  .solvers  tended  to  start  at 
the  beginning  of  the  example  and  read  until  they  found  a  useful 
line,  whereas  the  Good  solvers  started  reading  in  the  middle  of  the 
example  and  read  only  one  line. 

Similar  finditigs  have  also  been  observed  in  protocol  studies  of  stu¬ 
dents  learning  Lisjj  (Pirolli  Bielaczyc.  1989:  Bielaczyc  k.  Recker.  1991 ). 
elect rodynamics  ( Fergusson-Hessler  k  de  .Jong.  1990)  and  biology  (('hi. 
de  Leenw.  Chiu.  iC  La\'ancher.  1991).  This  cluster  of  findings  is  called 
the  self-exi)lanation  effect. 


THE  CASCADE  MODEL 

,A  consensus  has  emerged  in  both  machine  learning  and  cognitive 
psvchology  that  it  is  important  to  distinguish  two  kinds  of  learning: 

•  (Jne  kind  of  learning  is  responsible  for  getting  knowledge  from  the 
environment  into  the  mind  of  the  agent.  This  is  called  knowl¬ 
edge  acquisition  in  the  cognitive  skill  acquisition  literature  and 
knowledge-level  learning  in  machine  learning.  There  are  many 
possible  learning  processes,  depending  on  the  type  of  instructional 


infoniiatioii  available  in  the  environment  and  the  type  of  knowl- 
pdc*’  to  bo  ar<|niro(l.  The  Cascade  projorf.  for  instance,  focuses  on 
hou  agents  can  learn  college  physics  by  studying  worked  exaiiiple 
exercises  and  solving  problems. 

•  The  other  kind  of  learning  increases  the  effectiveness  of  knowledge 
that  is  already  in  the  mind  of  the  agent.  This  is  called  knowledge 
compilation  or  knowledge  tuning  in  the  cognitive  skills  literature, 
and  symbol-level  learning  in  machine  learning.  This  class  of  learn¬ 
ing  mechanisms  includes  explanation-based  learning  (EBL).  chunk¬ 
ing  (Newell.  1900).  production  composition  (Anderson.  1983).  and 
many  others.  Some  of  these  mechanisms  provide  e.xplanatiuns  for 
robust  findings  in  the  skill  acquisition  literature  (e.g.,  Anderson. 
19«7:  Newell.  1990). 

In  order  to  determine  whether  the  learning  in  the  Chi  et  al.  study 
was  knowledge  acciuisition  or  knowledge  compilation,  and  to  set  the  stage 
for  developing  a  simulated  student,  we  began  by  developing  a  problem 
solver  that  could  solve  the  problenrs  in  the  study.*  The  solver  was  based 
oil  past  mechanics  jirolilems  solvers  (Bundy  et  al..  1979;  Larkin.  1983: 
Novak  iV  Araya.  19X0)  as  well  as  our  informal  inspection  of  the  Chi  et 
al.  protocols.  The  resulting  solver  had  62  physics  rules  and  a  host  of 
mat  lu-matical  and  common  sense  rules.  These  62  rules  became  the  target 
ktiowledge  base.  The  first  goal  of  the  Cascade  project  was  to  understand 
when  they  were  learned  and  how. 

In  order  to  find  out  where  the  target  rules  could  be  learned,  two  peo- 
ph'  who  were  not  involved  in  the  development  of  the  knowledge  base 
(letertnined  whether  each  rtile  was  mentioned  anywhere  in  the  textbook 
])riot  to  the  point  where  the  examples  were  introducer!.  There  was  95'^7 
agreeiiK'tit  between  the  two  judges,  and  disagreements  were  settled  by 
a  third  judge.  I  hey  determined  that  oidy  29  of  the  62  rules  were  men¬ 
tioned  in  the  text.  The  other  33  rules  would  have  to  be  learned  during 
example  studying  or  problem  solving.^  This  indicates  that  knowledge 

'.Aclitallv.  it  could  solve  only  .J3  of  the  J.5  problems.  The  other  two  involved 
kiiienialirs  knowledge  that  we  did  not  bother  to  formalize.  These  two  problems  will 
be  ignored  throughout  the  remainder  of  the  chapter. 

hey  coidd  also  be  recalled  from  earlier  training  in  physics,  but  there  is  evidence 
I  list  ‘hi-:  seldom  occurred  I  Chi  k  Vanlehn.  1991:  VanLehn.  Jones  4’  Chi.  in  press). 


ar(|iiisii  ion  ninst  be  going  on  <Inring  example  stntlying  and/or  problem 
solving.  K’ now  l('(lgo  rompilation  alone  would  not  sufTice  to  explain  tlie 
sub  jec  ts’  learning. 

Cascade  models  two  basic  activities;  explaining  examples  and  solv¬ 
ing  problems.  Knowledge  acquisition  goes  on  during  both.  Because  the 
type  of  physics  problems  used  in  Chi's  study  involve  only  monotonic  rea¬ 
soning.  Cascade  uses  a  rule-ba.sed.  backwards  chaining  theorem  prover 
(similar  to  Prolog)  to  implement  both  activities.  A  physics  example  is 
jcresented  to  Cascade  as  a  set  of  propositions  representing  tlie  givens 
of  the  problem,  a  list  of  sought  quantities,  and  the  lines  of  the  prob¬ 
lem’s  solution.  For  instance,  the  example  of  Figure  1  is  represented  with 
the  information  of  'Fable  i.  Cascade  explains  each  line  by  proving  that 
it  follows  from  the  givens  and  the  preceding  lines.  To  solve  a  i)roblem. 
Cascade  is  presented  with  propositions  representing  the  problem's  givens 
and  is  asked  to  prove  a  proposition  of  the  form  “The  value  of  Q  is  X" 
for  each  sought  (piantity  Q.  In  the  process  of  proving  the  proposition. 
Cascade  derives  a  value  for  the  variable  .\.  thus  solving  that  part  of  the 
l>i(ib|em.  .Although  this  model  of  problem  solving  and  example  explain¬ 
ing  is  « learly  too  simi»le  to  cover  all  task  domains,  it  suflices  for  physics 
ami  olht'r  task  domains  dominated  by  monotonic  reasoning. 

Cascade  includes  two  kinds  of  analogical  problem  solving.  Both  tyjres 
of  analogy  Iregin  by  retrieving  an  exanijrle  and  mapping  the  example's 
gi\ens  to  the  current  proltlem's  givens.  These  retrieval  and  mai)i)ing  pro- 
( esses  usually  corres|)ond  to  overt  behavior.  The  subjects  Hip  through 
the  textbook  pages  in  order  to  locate  an  example,  then  look  back  and 
forth  between  the  example  and  problem,  comparing  the  diagram  and 
text  that  describe  the  example's  problem  with  the  diagram  and  text 
(h’scribing  the  inoblem  they  are  trying  to  .solve.  This  behavior  gener¬ 
ally  occurs  only  once  per  problem.  All  the  examples  and  most  of  the 
|)ioblenis  are  acromi)anied  by  diagrams,  and  usually  the  subjects  would 
search  for  an  analogous  example  after  looking  at  the  diagram  and  before 
reading  the  problem.  Thus,  we  think  that  the  major  process  of  retriev¬ 
ing  an  analogous  |)roblem  is  based  on  recalling,  finding  and  com|>aring 
diagrams.  'Fhis  retrieval  process  was  not  modeled  in  Cascade.  The  sys¬ 
tem  was  simply  told  which  examples  the  subjects  retrieved,  and  forced 
to  retrieve  the  same  ones. 


Problem;  The  figure  on  the  left  below  shows  a  block  of  mass  m  kept  at 
rest  on  a  smooth  plane,  inclined  at  an  angle  of  35  degrees  with  the 
horizontal,  by  means  of  a  string  attached  to  the  vertical  wall.  What 
are  the  magnitudes  of  the  tension  force  and  the  normal  force  acting  on 
the  block? 


Solution: 

(1)  We  choose  the  block  as  the  body. 

(2)  The  forces  acting  on  the  block  are  shown  In  the  free-body  diagram 
on  the  right. 

(3)  Because  we  wish  to  analyze  the  motion  of  the  block,  we  choose  ALL 
the  forces  acting  ON  the  block.  Note  that  the  blodc  will  exert  forces  on 
other  bodies  in  its  environment  (the  string,  the  earth,  the  surface  of 
the  incline)  in  accordance  with  the  action-reaction  principle:  these 
forces,  however,  are  not  needed  to  determine  the  motion  of  the  block 
because  they  do  not  act  on  the  block. 

(4)  Since  the  block  is  unaccelerated,  we  obtain: 

F-«-N-*-mg>0. 

(5)  It  is  convenient  to  choose  the  x-axis  of  our  reference  frame  to  be 
along  the  incline  and  the  y-axis  to  be  normal  to  the  incline  (see  figure 
above,  right). 

(6)  With  this  choice  of  coordinates,  only  one  force,  mg,  must  be 
resolved  into  components  in  solving  the  problem. 

(7)  The  two  scalar  equations  obtained  by  resolving  mg  along  the  x-  and 
y-axes  are: 

F  -  mg  sin  35  >  0  artd  N  -  mg  cos  35  ■  0. 

(8)  From  these  equations  F  arnj  N  can  be  obtained  if  m  is  given. 

Figure  1:  An  example 
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Tahlo  1:  All  English  version  of  tlie  representation  of  the  example  of 
Figure  1 

J’rohleiii  givens; 

The  current  situation  is  nametl  Lx. 
lx  is  a  staiulard-gravity  situation. 

Ulock-ix  is  a  block. 

String-ix  is  a  ina.ssless  string. 

Plane-ix  is  an  frictionless  inclined  plane. 

Ulock-ix  slides  on  Plane-ix. 

String-ix  is  tied  to  Block-ix. 

Ulock-ix  is  at  rest. 

Block-ix  is  above  Plane-ix. 

String-ix  is  above  Block-ix. 

String-ix  is  to  the  right  of  Block-ix. 

The  inclination  of  Plane-ix  is  35. 

The  inclination  of  String-ix  is  35. 

The  mass  of  Block-ix  is  m. 

Pi'oldem  sought s: 

Mie  magnitude  of  the  tension  (ovce  on  Block-ix  due  to  String-ix. 
riio  magnitude  of  the  normal  force  on  Block-ix  due  to  Plane-ix. 

Solution  lines: 

The  set  of  bodies  of  lx  is  Ulock-ix. 

The  set  of  arrows  on  the  free- body  diagram  for  Block-ix  is  {an  arrow 
at  inclination  35  ])ointing  up.  an  arrow  at  inclination  115  pointing 
up.  an  arrow  at  inclination  00  pointing  down}. 

The  set  of  axes  on  the  free-body  diagram  for  Block-ix  is  {an  x-axis 
at  inclination  35,  a  y-axis  at  inclination  115). 

The  magnitude  of  the  tension  force  on  Block-ix  due  to  String-ix  is 
0  -  0  +  ( 1(  mg)  sin(.35)). 

riie  magnitude  of  the  normal  force  on  Block-ix  due  to  Plane-ix  is 
t)  -  (  I  (  ntff  )  cost  .35  ( )  -f-  0. 
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OiiH  of  tlip  two  kinds  of  analogy  is  used  to  inak»’  search  control  deci¬ 
sions.  It  roiiK's  into  play  when  Cascade  has  two  or  more  rules  for  achiev¬ 
ing  a  goal  and  it  needs  to  select  aniong  them.  !l  uses  llie  analogical 
mapping  to  see  if  the  example’s  derivation  has  a  goal  that  is  equivalent 
to  the  goal  that  it  is  currently  working  on.  If  it  finds  an  equivalent  old 
goal,  the  rule  that  achieves  the  old  goal  is  chosen  for  achieving  the  new 
goal.  This  tyjie  of  analogy  is  called  analogical  search  control,  because  it 
uses  the  example  as  a  source  of  advice  on  which  of  several  alternatives 
to  try  nrst.  For  instance,  a  student  might  say.  “I  cannot  tell  whether  I 
should  project  this  onto  the  .\-axis  or  the  y-axis.  At  an  analogous  point 
in  the  example,  the>  projected  onto  the  ,x-a.xis.  so  Til  try  that  too." 
.\nalogical  search  control  is  also  used  in  the  Eureka  system  (.Jones. 
t  his  volume ). 

riie  second  type  of  analogy  is  used  when  Cascade  cannot  find  a  rule 
that  will  apply  to  the  current  goal.  It  uses  the  analogical  mapping  to 
(rv  to  find  a  line  in  an  old  exainjile  that  it  can  convert  into  an  appro- 
priaie  ride.  It  looks  foi  a  line  in  the  example’s  solution  that  mentions 
the  (  iirrent  goal  (nr  rather,  a  goal  equivalent  to  the  current  goal  under 
the  mapping).  .Most  lines  are  eriuations.  so  it  is  simple  to  convert  a 
liiK'  til  a  temporary  rule  which  ran  then  be  used  to  try  to  achieve  the 
goal,  lor  instance,  a  student  might  say.  “1  need  some  way  to  get  the 
tension  of  string  .\.  The  »xani|)le  has  a  line  saying  that  string  I’s  tension 
is  no/ sin( ;{()).  I  Imse  two  strings  are  analogous,  and  30  degrees  is  a;  al- 
ogous  to  -I'  degrees  in  this  problem,  so  I  bet  that  the  tension  of  string 
.\  is  mr;  sin(  I')."  This  type  of  analogy  is  called  transformational  anal¬ 
ogy.  after  a  similar  method  ext'lored  by  Carbonell  (19%).  As  C'arbonell 
discovered,  transformational  analogies  often  yield  wrong  answers. 

.A  ttia  jor  difference  lietween  the  two  kinds  of  analogy  is  that  analogical 
seati  h  control  refers  to  the  rules  that  achieved  goals  during  the  solution 
of  an  example,  w  hereas  transformational  analogy  only  refers  to  the  lines 
of  the  solution.  When  Cascade  explains  an  example,  it  stores  in  memory 
a  set  of  triples,  each  of  which  contains  the  example’s  name,  a  goal  and 
t  he  !  ide  I  hat  ai  hieved  it .  These  i  riples  are  what  analogical  search  cont  rol 
searches  through.  If  an  example  is  not  explained,  then  no  derivation  is 
recorded,  so  analogical  search  control  cannot  gel  any  advice  from  that 
exam|)le.  On  the  other  hand,  transformational  analogy  refers  only  to 
the  solution  lines.  These  are  present  regardless  of  whether  the  example 
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is  explained,  since  they  merely  reirreseut  what  tlie  student  can  see  as 
they  look  al  tin'  page  cnnlnining  the  example.  Thus,  transhnnintional 
analogy  can  lunction  even  if  the  example  has  not  been  explained. 

(  ascade  s  main  knowledge  accpiisition  method  is  called  explanation- 
based  learning  of  correctness  or  EBLC  (VanLelin.  Ball  Kowalski, 
1W)0).  I'he  basic  idea  is  to  divide  knowledge  info  domain  knowledge 
and  non-domain  knowledge.  Domain  knowledge  represents  rules  that 
the  student  believes  to  be  correct  and  appropriate  for  the  task  domain. 
Non-domain  knowledge  represents  rules  that  are  believed  to  be  incorrect 
or  relevant  only  toother  task  domains.  The  most  important  non-domain 
rules  for  learning  are  overly  general  rules.  They  can  apply  to  many  situa¬ 
tions.  but  they  often  draw  incorrect  conclusions.  For  instance,  a  domain 
rule  is  “If  there  is  a  tension  force  F  caused  by  a  string  S.  and  the  tension 
in  the  siring  is  T.  then  the  magnitude  of  the  tension  force  is  also  T." 
An  overly  general  rule  is.  “If  there  is  an  entity  F.  with  a  part  S.  and  a 
property  c>f  part  S  has  value  T.  then  a  property  of  the  entity  F  also  has 
value  T."  This  rule  hajrpens  to  be  a  generalization  of  the  domain  rule, 
but  as  argued  in  VanLehn  and  .Jones  (in  press),  not  all  domain  rules  have 
plausible  overly  general  counterparts. 

The  basic  idea  of  EBLC  is  to  use  overly  general  rules  whenever  domain 
rules  fail,  then  save  a  s|)ecialization  of  the  overly  general  rule  as  a  new 
domain  rule  if  all  goes  well,  for  instance,  the  domain  rule  just  mentioned 
is  learned  by  specialization  of  the  overly  general  rule.  EBLC  begins 
wlieji  Cascade  reacJjes  an  impasse  that  is  caused  by  missing  rules  in 
the  knowledge  base.  .An  imi)asse  is  defined  to  be  an  occasion  when  the 
current  goal  matches  none  of  the  known  domain  rules  or  problem  givens. 
Impasses  can  be  caused  by  missing  domain  knowledge  or  by  reaching  the 
end  of  a  dead  end  path  in  the  search  space  which  could  have  been  avoiderl 
by  making  a  belter  search  control  decision  earlier.  Cascade  explicitly 
checks  for  the  latter  [rossibility  before  deciding  that  an  impasse  is  caused 
by  missing  knowledge.  To  resolve  a  missing-rule  impasse.  Cascade  tries 
to  use  an  overly  general  rule  to  achieve  the  stuck  goal.  If  the  use  of  such 
a  nde  idlimately  leads  to  achieving  the  current  top  level  goal  (i.e..  to 
exirlain  a  line  or  to  find  the  value  of  a  sought),  then  Cascade  forms  a 
new  domain  rule  that  is  a  specialization  of  the  overly  general  one.  The 
specialization  is  chosen  so  that  it  is  also  a  generalization  of  the  particular 
usage.  For  instance,  on  one  inoblem  Cascade  could  not  determine  the 
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pressiiro  in  a  i)arf  of  a  containor  even  though  it  knew  the  pressure  in  the 
whole.  Sinre  there  was  no  alternative  solution  to  the  problem  using  its 
(loniaitt  rules.  Cascade  decided  that  it  was  at  a  niis.sing-rule  impasse.  It 
applied  the  o\erl\ -general  rule.  “If  an  object  is  composed  of  jrarts.  then 
the  property  values  of  the  |)arts  and  the  wholes  are  the  same.”  This  rule 
a|)pli(':ii ioti  ullitnately  led  to  a  solution  of  the  problem.  Cascade  then 
formed  a  new  domain  rnle.  “If  a  container  has  a  part,  then  the  pressure 
in  the  part  is  eipial  to  the  pressure  in  the  whole."  Thus,  Cascade  learned 
a  correct  rule  of  physics  by  specializing  an  overly  general  rule  in  order 
to  resolve  an  impasse  caused  by  missing  domain  knowledge. 

Cascade  has  a  second  technique  for  learning  new  rules.  It  applies  oidy 
when  it  is  explaining  an  example  and  attemirting  to  prove  a  proposition 
that  has  no  variables.  If  it  cannot  prove  the  proposition  with  either 
domain  rules  or  overly  general  rules,  then  it  gives  up  and  simply  accepts 
that  the  pro|)osilion  is  true.  It  also  builds  a  rule  that  sanctions  this  in 
future  similar  rases.  The  rules  say,  in  essence,  that  if  a  later  problem 
is  analogous  to  this  |)roblem.  then  the  analog  to  this  proposition  can 
be  assumed  true  for  that  problem  too.  This  type  of  learning  is  called 
analogical  abduction. 

From  a  machine  learning  )oint  of  view.  Cascade  does  both  knowledge- 
Iev<'l  learning  (via  EBFC  and  analogy  abduction)  and  symbol-level  learn¬ 
ing  (via  the  saving  of  derivations,  which  are  used  by  analogical  search 
<(intr(j|).  FIILC  and  analogy  abduction  are  both  triggered  by  impasses, 
so  they  will  often  be  referred  to  as  imi)asse-driven  learning. 

.\s  a  sumtnary.  Table  2  lists  Cascade ‘s  main  processes.  Notice  that  a 
new  one  has  been  slipped  in.  Cascade  can  be  told  to  ignore  an  example 
line  instead  of  self-explaining  it.  a  trivial  process  labeled  “accent ance" 
in  t  he  table. 

Cascade's  learning  is  similar  to  those  propo.sed  by  existing  ries 
of  skill  ac(|uisit ion.  \^'e  believe  that  analogical  search  control  can  even¬ 
tually  i)rovide  an  account  for  the  practice  effects  usually  explained  by 
knowledti*'  compilation  (Anderson.  1083).  chunking  (Newell.  1990)  and 
other  learning  mechanisms.  EBLC  is  similar  to  proposals  by  Schank 
(19<'<(»).  Lewis  (1988),  Anderson  (1990)  and  others,  which  also  acquire 
new  knowledge  at  impasses  by  specializing  existing,  overly  general  knowl¬ 
edge.  .Although  all  these  models  of  skill  acquisition  are  similar  in  spirit, 
thev  differ  in  significant  ways.  For  more  on  the  Cascade  system  and  a 
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Tal)le  2:  Cascade's  major  processes 

Example  studying 

•  Sell-explanation:  Prove  a  line  via  backwards  chaining. 

•  Acceptance:  Ignore  (lie  example  line. 

Problem  Solving 

•  Regular  problem  solving:  Find  a  value  for  a  sought  via  backwards 
chaining.  At  search  control  choice  points,  use  analogical  search 
control  to  decide  which  rule  to  apply. 

•  Iranshuinational  analogy:  Find  a  line  in  an  example  that  could  be 
adapted  to  achieve  the  current  goal. 

1  111 |)asse-d riven  learning 

•  E.xplanation-based  learning  of  correctness  (EBLC):  Apply  an 
overly  general  rule.  If  that  leads  to  success,  save  a  specialization 
as  a  new  domain  rule. 

•  .Analogy  abduction:  Like  transformational  analogy,  except  a  rule 
is  liiiilt  so  that  future  occurrences  of  the  goal  will  be  handled  the 
same  way. 
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(Iflailc'il  comparison  with  its  predecessors,  see  VanLehn  and  Jones  (in 
prc'ss ). 


MODELING  THE  SELF-EXPLANATION  EF¬ 
FECT  WITH  CASCADE 

A  simple  liypot  liesis  for  explaining  the  four  major  differences  between 
(iood  and  Poor  solvers  is  that  Good  solvers  chose  to  explain  more  ex¬ 
ample  lines  than  Poor  solvers.  To  test  this,  several  simulation  runs  were 
made.  .All  these  simulations  bc'gan  with  the  same  initial  knowledge.  The 
initial  domain  knowledge  consisted  of  the  29  physics  rules  that  three 
judges  fonnd  to  be  jrresenf  in  the  text  (see  the  discu.ssion  at  the  begin¬ 
ning  of  the  precc'ding  section).  The  rest  of  the  initial  knowledge  base 
consists  of  -l-'j  non-domain  rules,  of  which  28  represented  common  sense 
physics  (e.g..  a  taut  rope  tied  to  a  object  pulls  on  it)  and  17  rejrresented 
over-generalizations,  such  as  “If  there  is  a  push  or  a  pull  on  an  object  at 
a  ceriain  angle,  then  there  is  a  force  on  the  object  at  the  same  angle." 
See  \'anl.<'hn.  .lones  and  ('hi  (in  press)  for  a  list  of  the  overly  general 
rules. 

In  principle.  Cascade  can  use  regular  irroblem  solving  or  transforma¬ 
tional  analogy  at  any  goal.  For  the  sake  of  these  experiments,  we  gave 
it  a  fixed  strategy.  It  would  first  try  regular  problem  solving.  If  that 
failed  dm'  to  missing  domain  knowledge,  then  impasse-driven  learning 
was  applied.  Transformational  analogy  was  used  only  as  a  la.st  resort. 

The  first  simulation  was  intended  to  model  a  very  good  student  who 
explains  every  line  of  every  example.  Cascade  first  explained  the  3  ex¬ 
amples  in  the  study,  then  it  solved  the  23  problems.  (The  2  itroblems 
that  are  not  solvable  by  the  target  knowledge  were  excluded.)  It  was 
able  to  correctly  solve  all  the  problems.  It  acrpiired  23  rules:  8  while 
ex|)laining  examjrles  and  15  while  solving  problems.  .All  but  one  of  the 
rules  was  learned  by  EBLC:  analogical  abduction  learned  the  other.  The 
new  rid*'s  are  correct  |>hysics  knowledge,  allowing  for  the  simirlicity  of 
the  knowlerlge  representation.  .Moreover,  they  seem  to  have  the  right 
degree  of  generality  in  that  none  were  apjrlied  incorrectly  and  none  were 
inap|)licable  wdien  they  should  have  been  applicable.  However,  some  of 
the  rules  dealt  with  situations  that  only  occurred  once  in  this  problem 


14 


sot.  so  they  wpip  never  used  after  their  acquisition. 

I'lx'  sorninl  sinnilation  was  intended  to  siinnlato  a  very  poor  student 
wlio  does  no  self-e.Kplanation.  Because  none  of  example  lines  were  ex¬ 
plained.  there  was  no  opportunity  for  EBLC  to  learn  new  rules  during 
example  studying,  nor  were  any  derivations  left  behind  for  use  by  analog¬ 
ical  s('arch  control  during  later  problem  solving,  (.'ascade  was  given  the 
saiiK'  2d  problems  given  to  it  in  the  good  student  simulation.  It  correctly 
solved  tj  problems.  .Apparently  these  problems  require  only  knowledge 
from  the  text.  .As  Cascade  solved  these  problems.  Cascade  learned  -i  cor¬ 
rect  rules  via  EBLC.  On  (i  other  problems.  Cascade  found  an  incorrect 
solnliou.  f!BLC  did  not  occur  on  these  problems.  On  the  remaining  S 
])roi)iems.  Cascade  failetl  to  find  any  solution  or  its  search  uent  on  for 
so  lone,  that  it  was  cut  it  off  after  20  minutes.  .Although  EBLC  was  used 
extensively  on  these  |)roblems.  the  rules  producerl  were  always  incorrect. 
On  the  assumption  that  a  poor  student  would  not  believe  a  rule  uidess  it 
led  to  a  correct  solution  to  a  i)roblem.  ndes  acrpiired  during  failed  solu¬ 
tion  attempts  were  deleted.  Thus,  the  poor  student  simulation  acrpiired 
only  .1  rules  and  solved  only  9  problems  correctly. 

Explaining  the  self-explanation  correlations 

( ■as<  ade  should  be  able  to  explain  the  four  differences  observed  by  Chi 
el  al.  (Idsft)  hetween  Cood  and  Poor  solvers.  .Assimiing  that  the  num¬ 
ber  of  self-explanatory  utterances  is  directly  proirorl ional  to  the  nnndrer 
of  lines  explained  during  example  studying,  the  job  facing  Cascade  is 
to  explain  why  explaining  more  lines  causes  better  .scores  on  quantita¬ 
tive  post  -  test  s  ( fimling  1 ).  more  accurate  self- monitoring  ( finding  2 )  and 
more  fretpient  (  finding  d )  and  more  economical  reference  to  t  he  examples 
I  finding  I ). 

The  coni  fast  beiweefi  the  good  and  poor  student  simulations  indi¬ 
cates  that  Cascade  can  reproduce  the  positive  correlation  between  the 
numbei  of  example  lines  exiilained  and  the  number  of  problems  solved 
(orrecily.  During  t  he  good  student  simulation,  it  exjilained  all  the  exam¬ 
ple  lines  and  got  all  T-i  problems  correct;  on  the  poor  student  simulation, 
it  explained  none  of  the  example  lines  and  got  9  of  the  problems  correct. 
Knowing  the  operation  of  Cascade,  it  is  clear  that  having  it  explain  an 
intermeriiale  number  of  lines  would  cause  if  to  correctly  answer  an  in- 


tennodiate  luiiiiber  of  problems.  So  the  two  extreme  points  (the  two 
simnlalions)  |>bis  Cascade's  dclerminislir  design  are  su/ricient  to  demon- 
sfrale  tlie  main  finding  of  the  self-explanation  effect. 

One  of  the  major  advantages  of  a  simulation  like  Cascade  is  that  one 
can  run  it  many  times  with  dilferent  components  lurnetl  off  in  order  to 
ascertain  why  it  succeeds.  In  particular.  20  rules  were  learneti  by  the 
good  student  simulation  and  not  by  the  poor.  For  each  rule,  we  can  find 
out  why  self-ex|)lanalion  allowed  Cascade  to  learn  it. 

First,  when  more  lines  are  explained.  Ca-scade  is  more  likely  to  stum¬ 
ble  across  a  gap  in  its  domain  knowledge.  Such  missing  knowledge  causes 
impasses,  which  lead  to  impasse-driven  learning  and  the  ac(|uisition  of 
new  rules  during  exami)le  explaining.  Of  the  20  rules  that  were  learned 
dnritig  the  good  student  simulation  and  not  the  ]>oor.  (40/^)  were 
lei  i  lied  while  explaining  examples. 

Analogical  search  control  also  aided  the  good  student  simulation  s 
h'arning.  When  more  lines  are  explained,  more  derivations  become  avail- 
aide  for  analogical  search  control.  Analogical  search  control  lends  to  keep 
(  ascade  on  solution  paths  during  problem  solving,  and  this  means  that 
an\  impasses  that  occur  ate  more  likely  to  be  due  to  missing  domain 
knowledn*'.  1  hits.  EUI.(  is  more  often  applied  to  appropriate  impasses, 
and  iliiis  inoie  often  generates  correct  domain  rules.  Of  the  20  rules.  0 
(  I't'/i  I  recpiire  analogical  search  control  for  their  acquisition. 

1  he  accinisition  of  rules  during  example  studying  helps  produce  con¬ 
texts  during  problem  solving  that  allow  EBLC  to  leant  tnore  rules  during 
problem  solving  even  without  the  aid  of  analogical  search  control.  Of  the 
10  rules.  4  (  I'tVi  )  can  be  acquired  during  problem  solving  even  when  ana¬ 
logical  search  control  is  turned  off.  These  new  rules  also  contributed  to 
the  improvement  in  problem  solving.  Table  3  summarizes  the  learning 
of  t  he  I  wo  runs. 

(  asc  ade  provides  a  simple  explanation  of  the  correlation  between  the 
amount  of  scdf-explanat ion  and  the  accuracy  of  self-monitoring  state¬ 
ments.  The  explanation  assumes  that  negative  self-monitoring  state¬ 
ments  (e.g..  ”1  don't  understand  that")  correspond  to  impasses,  and 
that  i)ositive  self-monitoring  statements  (e.g..  '‘Ok.  got  that.")  occur 
with  some  probability  during  any  non-inipasse  situation.  When  more 
example  lines  are  explained,  there  are  more  impasses,  and  hence  the 
|)ropoiiic)n  of  negative  self-monitoring  statements  will  be  higher.  In  the 


T.il)l(‘  3:  Hulos  lonniod  during  Goo<l  and  Poor  stiulent  simulations 


>.iood  Poor  Wlion  ar((uired 
s  0  Example  studying 

Prol)lem  solving 

:J  ;{  No  ex.  studying  rules,  no  analogical  search  control 

U  With  ex.  studying  rules,  no  analogical  search  control 

<1  0  With  ex.  studying  rules,  with  analogical  search  control 

2'\  3  Total 


extreme  case  of  the  p<H)r  student  simulation,  where  no  exam])le  lines  are 
explained,  all  the  self-monitoring  statements  during  example  processing 
would  he  i)ositive.  wliidi  is  not  far  off  from  Chi  et  al.'s  observation  that 
S')','  of  the  Poor  solver's  self-monitoring  statements  were  positive. 

I  he  third  and  fourth  findings  involve  the  frequency  and  specificity  of 
analogi(  al  references  during  problem  solving.  The  number  of  references 
made  by  analogical  search  control  and  transformational  analogy  were 
counted.  We  assumed  that  only  some  of  the  analogical  search  control 
|■('fe^ences  to  t lie  derivation  were  overt,  and  that  the  others  were  mental 
refeD'uros  that  would  not  show  uj)  in  the  Chi  et  al.  data.  This  gave  us  a 
liredictioM  of  the  freciuency  of  analogical  references.  To  get  a  prediction 
of  the  specificity  of  atialogical  references  (i.e..  the  number  of  example 
lines  read  jier  referetice ).  we  counted  the  number  of  lines  read  by  tratis- 
formational  analogy  before  it  found  one  it  could  use.  and  we  assumed 
t  hat  someone  using  analogical  search  control  would  go  directly  to  the  line 
whose  derivation  contained  the  sought  goal.  Given  these  assumirtions. 
tin'  good  student  simulation  produced  fewer  and  more  specific  analogical 
rr-feiemes  than  the  poor  student  simulation,  thus  modeling  the  Chi  et 
al.  finding  (see  \  anLehn.  .lones  (C  Chi.  in  press,  for  details). 

Discussion 

Although  we  controlled  Cascade’s  behavior  during  example  studying, 
bv  either  telling  it  whether  to  explain  the  examples  or  not,  its  behavior 
during  |>roblem  solving  was  determined  solely  by  how  much  it  learned 
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(luiiiif!,  exampU*  studying.  Qualitatively,  the  behaviors  of  the  Good  and 
1*001  runs  \v('i(’  (|uit<>  similar  to  the  behaviors  of  the  Good  ami  Poor 
students  during  problem  solving.  The  good  student  simulation  tended 
to  stay  on  solution  i)aths.  use  regular  problem  solving  more  often  than 
transformational  analogy,  and  learn  something  from  the  orra.sional  itti- 
passf's  it  encountered.  The  Poor  student  simulation  tended  to  wander 
(low  n  unproductive  jraths.  use  transformational  analogy  more  often,  and 
leam  not  hint*  from  the  many  impasses  that  it  encountered. 

These  inoperties  of  Cascade's  problem  solving  behavior  are  consistent 
with  a  preliminary  analysis  by  Chi.  VanLehn  fc  Reiner  (19.*^^^).  who  ana¬ 
lyzed  the  protocols  of  a  Good  solver  and  a  Poor  solver  as  they  solved  the 
same  problem.  The  Poor  solver's  protocol  was  divided  into  77  episodes, 
and  of  these.  dO  (d!)*7i  )  resulted  in  impasses.^  Many  of  these  impasses 
s('(>med  to  result  in  acquiring  incorrect  beliefs.  In  contrast,  the  protocol 
of  the  Good  solver  was  divided  into  31  episodes,  of  which  only  7  (‘id'/I ) 
(•('sidted  in  imi>;isses.  In  ()  of  these,  the  Good  solver  seemed  to  learn 
a  ((UK^ct  piece  of  knowledge.  This  preliminary  analysis  indicates  that 
the  Poor  s(dvers  had  proportionally  more  impasses  (39‘7(  )  than  the  Good 
solvers  Cid'.f)  while  problem  solving,  and  that  the  resrdting  knowledge 
was  more  oft(Mi  incorrect.  This  is  just  what  Cascade  did.  too. 

r.xami)|e  studying  took  U|>  a  relatively  small  proportion  of  the  time 
that  suljjects  spent  durine,  the  study.  Not  only  were  there  only  3  exam- 
ples  rom|)ared  to  2^)  i)roblems.  the  subjects  spent  less  time  on  average 
studyine,  ati  exam])le  thati  solving  a  problem.  The  learning  strategy  of 
self-explanation  was  active  only  during  example  studying,  .so  it  comes  as 
a  surprise  that  such  a  proportionally  small  change  in  work  habits  caused 
such  a  laiKe  chattge  itt  the  amount  learned.  Perhaps  the  most  important 
result  from  the  Good/Poor  simulations  is  an  explanation  for  this  coun- 
lerintiiitire  findine.  The  simulations  showed  that  only  40/1  of  the  rtih's 
learned  by  the  good  student  simulation  and  not  Iry  the  poor  were  learned 
during  example  studying.  The  others  were  learned  during  problem  solv- 
inu.  This  came  as  somewhat  of  a  surprise  to  us.  There  were  two  basic 
reasnus  that  self-explanation  increases  learning  durinp,  problem  solving. 

All  iiiipa.«s»>  Wiu!  identified  as  an  outcome  of  an  episode  whenever  the  student 
lielieves  that  the  next  step  that  should  lie  executed  cannot  be  performed.  Most  (9St^) 
of  the  impasses  were  identified  by  explicit  statements  such  as  “I  don't  know  what  to 
do  with  thf  aiiKle  "  or  “So  that  doesn't  work  either." 


•  Tlie  lilies  leanieil  earlier  allowed  Cascade  to  travel  down  correct 
soliilioii  jiallis  and  reacli  impasses  at  places  were  it  was  indeed 
missing  knowletlge.  Without  these  rules,  the  poor  student  simula¬ 
tion  could  not  reach  these  productive  impasses. 

•  The  derivational  triples  acipiired  during  rederivation  of  the  exam¬ 
ple  lines  served  as  search  control  advice  during  problem  solving, 
thus  tending  to  keej)  the  good  student  simulation  on  solution  paths 
that  led  to  produc'ive.  missing-knowledge  impasses.  The  poor  stu¬ 
dent  simulation  tended  to  wander  off  the  solution  jiaths.  and  reach 
impasses  where  there  was  nothing  valuable  to  be  learned. 

It  is  doubtful  that  these  interactions  would  have  been  discovered  without 
a  simulation  as  detailed  as  Cascade. 

These  results  taught  us  about  self-explanation  jier  se.  but  the  use  of 
idealized  student  simulations  leaves  open  the  question  of  whether  Cas¬ 
cade  can  actually  model  a  real  student.  The  next  study  tackles  this 
(piesi  ■■m. 


MODELING  THE  PROTOCOLS  OF  INDIVID¬ 
UAL  SUBJECTS 

Tlie  ob  jective  of  the  study  reported  in  this  section  was  to  find  out  how 
clos(>  ( 'ascad<'  can  come  to  modeling  individual  subjects.  This  study  was 
nnderiaken  in  the  same  spirit  as  the  ones  in  .VewelJ  and  Simon  (1072): 
(dven  a  protocol,  how  closely  can  a  simulation  be  fit  toil?  One  difference 
between  this  study  and  those  of  Newell  and  Simon  is  that  the  task  domain 
is  jiliysics.  wliidi  is  arguably  a  much  richer  task  domain  than  the  ones 
ihe\  studied.  However,  a  more  important  difference  is  that  considerable 
learning  look  jilaie  during  our  protocols. 

.\  third  difference  is  that  our  jirotocols  are  much  longer  than  Newell 
and  Simon's  protocols,  which  made  it  impossible  toemploy  their  method 
of  anah  sis.  Facli  of  the  0  subjects  coni rilmted  protocols  for  3  exanijiles 
and  2'i  problems,  so  there  were  252  protocols  to  analyze.  Each  jirotocol 
averaged  about  12  jiages.  for  a  total  of  .1000  pages.  Creating  problem 
behavior  graphs  for  all  of  them  would  be  far  too  much  work.  Thus,  part  of 
the  rliallenge  in  this  study  was  to  devise  feasible  methods  for  measuring 
the  match  between  the  behaviors  of  Cascade  and  the  subjects. 
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Fignro  2:  Matfhiug;  the  behaviors  of  Cascade  and  a  sulrject 

Fip,me  2  sliows  how  the  match  between  the  behaviors  of  Cascade  and 
a  sul)ject  can  be  viewed.  Region  1  represents  behavior  that  the  subject 
e.xhibiied  and  Ca.scade  did  not.  Region  2  represents  the  behaviors  that 
are  the  same  for  both  agents.  Region  3  represents  Cascade  behaviors  that 
I  lie  subject  did  not  e.\hil)it.  The  behaviors  in  region  3  have  two  sources. 
Some  are  computational  e.xpediencies:  VVe  couldn't  get  Cascade  to  do 
exacllv  what  the  suliject  did.  so  we  had  it  do  something  else  instead. 
That  “something  else"  shows  up  in  region  3.  A  Cascade  behavior  will 
also  be  put  in  regiott  3  if  it  is  plausibly  something  that  the  subject 
did.  but  the  protocol  happens  to  show  no  signs  of  it  occurring.  For 
instance,  it  is  known  that  not  all  ca.ses  of  impasse-driven  learning  show 
up  as  hesitations  or  negative  comments  in  protocols  (VanLehn.  1991a). 
Whenever  Cascatle's  impasse-driven  learning  is  not  reflected  by  overt 
signs  of  an  impasse  in  the  subject's  protocol,  that  behavior  is  classified 
as  region  3  behavior. 

In  all  of  the  analyses  presented  below,  we  tried  to  determine  two 
ratios:  the  amount  of  Ca.scade  behavior  that  is  matched  by  the  subject 
(region  2  divided  liy  the  union  of  regions  1  and  2).  and  the  amount  of 
subject  behavior  that  is  matched  by  Cascade  (region  2  divided  by  the 
unions  regions  2  and  3).  In  order  to  make  these  comparisons,  we  had  to 
find  a  way  to  count  behaviors,  which  implies  choosing  a  unit  of  analysis. 
This  was  not  hard  for  the  first  ratio,  because  Cascade's  behavior  is  well 
defined.  For  instance,  we  usually  used  a  goal  as  the  unit  of  analysis  and 
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connttMl  tlip  iniml)er  of  goals  generated  by  Cascade  that  were  iiiatclied 
nr  iinmalchod  liy  llie  subject.  It  was  not  easy  to  determine  a  unit  of 
analysis  for  the  other  ratio,  the  percentage  of  subject  behavior  matched 
In  Cascade.  A  variety  of  units  were  used,  depending  on  the  type  of 
analysis  being  conducted. 

1  ive  analyses  were  conducted  {see  Table  4).  Because  we  are  more 
interested  in  getting  Cascade  to  simulate  the  subjects'  accpiisition  of 
physics  rules  than  in  getting  it  to  simulate  the  chronology  of  their  rea¬ 
soning.  four  of  the  analyses  ignored  the  order  in  which  Ca.scade  and 
the  subject  made  inferences.  Both  Cascade's  behavior  and  the  subject's 
behavior  were  reduced  to  sets  of  inferences.  Set  intersections  and  ditler- 
eiices  were  calculated,  just  as  shown  in  Figure  2.  However,  we  cannot 
entirely  ignore  the  chronology  of  inferencing.  since  the  earlier  study  in¬ 
dicated  that  analogical  search  control  affects  the  location  of  impasses, 
which  in  turn  determines  what  can  be  learned  during  problem  solving. 
■So  a  fifth  analysis  was  conducted  in  order  to  see  if  the  subjects'  choices 
diiiiutt  prolr|pi\i  solving  could  lie  predicted  by  analogical  search  control. 
Afl''!  a  description  of  how  Cascade  was  fitted  to  the  jrrotocols.  each  of 
these  unalyses  will  be  presented. 

Fitting  Cascade 

Fitting  Cascade  means  setting  values  for  parameters  so  that  the  pro¬ 
gram's  beliavior  matches  the  given  subject's  behavior  as  clo.sely  as  pos- 
sible.  The  parameters  represent  the  products  of  cognitive  processes  that 
are  not  modeled  by  Cascade,  and  yet  Cascade's  performance  deirends 
on  the  outputs  of  these  unmodeled  processes,  so  they  cannot  be  ignored 
ent  irely. 

I'here  are  two  major  types  of  parameters.  The  first  controls  initial 
knowU'dgp.  which  refers  to  the  knowledge  possessed  by  a  student  or  Cas- 
cadf  just  prior  to  studying  the  e.xamples.  The  student's  initial  knowledge 
(omes  from  reading  the  first  several  chapters  of  the  le.xtbook  and  from 
their  earlier  stiulies  of  physics  and  mathematics.  F'ascade  does  not  model 
these  processes,  so  it  must  be  given  an  initial  knowledge  base.  Cascade's 
initial  knowledge  base  was  alw'ays  a  subset  of  a  fixed  “rule  library."  The 
library  consists  of  3  buggy  physics  rules."'  the  62  rules  that  constitute 

’Onr  KngRv  rule  appliec  F  =  rnn  to  any  force  and  not  just  a  net  force  Another 
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TiiMo  I:  Aiialysos  romparing  f'asradr's  boliavior  to  the  snbjc'rls*  behav¬ 
ior 


1.  How  many  of  Cascade's  example  studying  inferences  were  also 
made  lyv  the  sul)jert? 

2.  How  many  of  the  subject's  example  studying  inferences  were  also 
made  by  (  ascade? 

■i.  How  many  of  Cascade's  problem  solving  inferences  were  also  made 
by  I  he  subject  ? 

t.  How  many  of  the  subject's  problem  solving  inferences  were  also 
made  by  Cascade'.' 

■>.  Do  the  search  control  decisions  made  by  the  subject  match  those 
made  by  (  ascade? 


the  target  domain  knowledge,  and  the  do  non-domain  rules  mentioned 
earliei.  .S('|e(  iing  an  initial  knowledge  ba.se  can  be  viewed  as  .setting  IJO 
binai  v  parameters,  one  for  each  rule  in  the  librarv.  where  1  means  that 
the  rule  is  included  in  the  initial  knowledge  base,  and  0  means  that  the 
i  nle  is  e\<  bided. 

I  he  second  type  of  parameter  controls  the  depth  of  seif-exi)ianation. 
Wlimi  sindving  examples.  snl)jects  choose  to  explain  some  Lines  but  not 
others.  Fven  when  they  do  explain  a  line,  they  may  explain  it  only 
down  to  a  certain  level  of  detail  and  decide  to  take  the  example's  word 
lor  the  rest.  For  instance,  they  might  explain  most  of  the  line.  F.,,  = 
-  r.,  cost :}()).  but  not  bother  to  explain  where  the  minus  sign  comes  from. 
(  ascade  does  not  model  how  the  subjects  decide  which  lines  to  ex]>lain 
and  ho«  deeply  toe.xplain  them.'’  To  simulate  the  output  of  this  decision 

that  llif  ina«c  of  a  hotly  is  etiual  to  its  weij^ht.  The  third  assumes  that  the 
siRii  of  all  project  ions  is  positive 

''There  are  many  possible  reasons  for  why  subjects  do  not  explain  everything.  For 
instance,  the  subjects  may  feel  that  they  already  know  everything  that  they  could 
learn  from  explaining  the  line,  or  they  may  feel  that  explaining  such  details  can  be 
left  until  such  lime  as  they  really  need  to  know  them.  Deciding  how  deeply  to  explain 
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iiiakiiij:,  itKxc^ss.  extra  jnoposif ions  were  entered  into  the  descriptions  of 
examples.  Wlienover  Cascade  is  ahoni  to  ex|)lain  something,  it  first 
checks  to  see  if  accept(P)  is  in  the  example's  description,  where  P  is 
file  thing  it  is  about  to  explain.  If  an  accept(P)  is  foujid.  Cascade 
meit'l\  aicepls  P  as  explained  without  an.v  further  processing.  V'iewed  as 
parameter  setting,  this  amounts  to  as.sociating  a  binary  parameter  with 
every  ex|)lainable  oirject.  and  setting  it  to  1  if  it  should  be  explained  and 
0  if  it  should  be  accepted. 

The  accejrt  proj)Ositions  are  set  by  inspecting  the  subject's  protocol. 
If  the  subject  merely  reads  a  line  and  says  nothing  else  about  it.  then 
an  a<  (  ept  proposition  is  entered  for  the  whole  line.  If  the  subject  omits 
dis(  iission  of  a  detail  in  a  line,  then  an  accept  is  placed  around  the 
( 'as(  ade  goal  that  coi  responds  to  that  detail.  In  this  fashion,  the  data 
com|)l('lely  determine  which  lines  and  parts  of  lines  are  explained  by 
( 'as(  ade. 

On  th<'  other  hand,  there  is  no  way  to  easily  determine  what  the 
subject ‘s  initial  knowlerlge  is.  The  whole  protocol  must  be  examined.  .As 
will  !)('  seen  later,  we  sometimes  made  mistakes  in  selecting  the  initial 
knowh  ilgo.  We  should  have  fixed  our  mistakes,  rerun  the  simulations 
ami  redone  the  comparisons  of  the  program's  output  with  the  protocols. 
1  his  will  re(iuire  months  of  work,  so  for  this  chapter,  we  are  forcetl  to 
la-porl  the  analyses  with  our  imperfect  choices  of  initial  knowledge  left 
intact . 


How  many  of  Cascade’s  explanations  are  matched? 

riiis  section  discusses  the  behavior  of  Cascade  and  the  subjects  as 
they  explained  exatititles.  Goals  were  used  as  the  basis  for  dividing  Cas- 
cade  s  beliaviot  into  countairle  units.  Each  goal  produced  by  Casca<le 
was  classified  accorditig  to  the  method  used  to  achieve  it; 

•  Hee,itlar  e\])lanation:  Cascade  used  one  of  the  domain  rules. 

•  Impasse  with  learning;  Cascade  reached  an  impasse,  successfully 
applied  an  overly  general  rule,  and  learned  a  new  domain  rule  via 
EBLC  or  analogical  abduction. 


a  line  is  a  fa.«riiia(inR  topic  for  future  research. 


•  Acropt  witlioiit  ex|)laiiation:  The  goal  was  not  processed  any  fnr- 
t  h<’i .  Init  inerf'ly  arropled  as  true  willtoni  proof,  hecanse  tlie  e.x- 
aiiiple's  (lesf ripl ion  contained  an  "accept"  proposition  for  it. 

.Agerettat iiiK  across  the  siimilatiou  runs  of  all  9  subjects,  there  were 
1121  goals.  W'e  located  each  of  these  goals  in  the  subjects'  protocols, 
and  based  on  I lu’  talk  surrounding  them,  classified  them  into  the  same 
throf  categories  plus  a  new  one: 

•  lm|)asse  with  learning:  If  the  subject  paused,  complained  about 
the  goal  nr  in  some  other  way  showed  ngns  of  being  stuck,  then  we 
classified  the  goal  as  being  achieved  by  impasse-driven  learning. 

•  Regular  explanation:  If  the  subject  merely  nientit>ned  the  goal  oi 
its  (omlusions  without  any  fuss,  or  the  sul)ject  said  nothing  at  all 
about  this  goal  but  dirl  mention  its  subgoals,  then  we  classified  the 
goal  as  being  solved  liy  regular  exi)lanation. 

•  ,\((ept  without  exi)lanation:  If  the  subject  said  not  lung  about  this 
glial  iioi  its  subgoals,  then  the  goal  was  classified  as  being  accepted 
u  ii  lioiil  explanation. 

•  Impasse  and  accepi  ;  Sometimes  subjects  clearly  tried  to  explain  a 
goal,  but  couldn't  doit  at  all.  so  t  hoy  just  accepted  the  goal  wit  hout 
|)roof.  This  is  tlifferent  from  the  other  kind  of  acceptance,  where 
the  subject  did  not  even  try  to  explain  the  goal.  It  is  different  from 
iIh‘  other  kind  of  im|)asse  because  no  learning  occurs. 

fable  'i  shows  tin'  I  121  goals  and  how  they  were  classified.  Most 
(  lObl  =9  +  2-1-  ()')  I  +  d9(i)  goals  were  processed  the  same  way  by  both 
(  ascade  and  the  subject,  so  95M  of  Cascade's  behavior  was  matched  by 
subject  behavior,  w  hich  is  highly  significant  {p  <<  .001.  ( 'hi-s{piared 
test).  In  order  to  get  a  cpialitative  understanding  of  the  shoitcomings 
in  (  ascade's  model  of  the  protocols,  each  of  the  off-diagonal  cells  is 
discussed . 

There  were  7  cases  where  Cascade  learned  a  rule  and  the  subjects 
were  coded  as  accepting  the  goal.  All  7  cases  occur  at  the  same  point, 
on  a  line  w  here  the  exam[»le  says.  "Consider  the  knot  at  the  junction  of 
the  three  string-  to  be  the  hotly."  Explaining  this  line  causes  Cascade 
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Tnlik  •'):  Pxipoi I ijtiis  of  fasrado  artioiis  mafrlicd  by  siibjf'ot  actions 


Ca*cad* 

Subiaeta  I 

Impasse 

unresolved 

Regular 

•xDlaraiion 

Accept 

silently 

Totals 

Impasse  *  EBLC 

e 

0 

10 

1  0 

Impasse  *  Anal.  Abdud. 

0 

2 

0 

0 

Regular  explanalion 

0 

0 

6S4 

603 

Accept  w/o  explanalion 

0 

4 

0 

306 

400 

Totals 

e 

6 

664 

442 

1121 

to  learn  a  new  rule  via  analogical  abduction.  However,  only  2  of  the  9 
snl)jects  coniiuented  about  this  rule  during  example  studying.  The  other 
7  said  nothing  at  all  about  the  line  wherein  this  rule  would  be  learned, 
so  they  were  coded  a.s  accei)ting  the  goal  without  proof.  We  could  have 
nia<le  ( 'ascade  accept  the  goal  as  well,  which  meant  that  it  wouldn't  learn 
the  knot  rule.  During  later  problems  that  had  three  strings  converging 
on  a  knot,  this  would  cause  Cascade  to  reach  an  impasse  and  use  trans¬ 
formational  analogy.  Unfortunately.  Ca.scade's  transformational  analogy 
mechanism  is  not  powerfid  enough  to  make  use  of  the  knot-is-a-body 
line  in  the  exam))le.  When  we  increased  its  power  so  that  it  could  use 
this  line,  it  became  too  powerful  and  would  draw  analogies  that  were 
so  far  fetched  that  no  subject  would  consi<!er  them.  This  led  us  to  in¬ 
vent  analogical  abduction,  which  is  a  novel  type  of  machine  learning  (see 
VaiiLehn  <V  .lones.  in  press,  for  discussion).  In  order  to  test  it  out.  it 
was  included  in  Cascade.  However,  it  is  clear  from  this  analysis  that 
tliere  are  empirica)  problems  with  it.  If  an  example's  line  really  does 
cause  analogical  abduction,  which  is  a  form  of  impasse  driven  learning, 
then  more  subjects  should  have  shown  impasses.  We  now  believe  that 
transformational  analogy  is  actually  the  source  of  transfer  between  the 
e.xample  line  and  problem  solving,  and  that  the  two  subjects  who  had 
impasses  here  should  be  classified  as  “imijasse  and  acce])t."  rather  than 
as  learning  a  new  rule.  As  will  be  seen  later,  there  are  other  signs  that 
Cascade's  model  of  transformational  analogy  is  flawed. 

There  were  10  cases  where  Cascade  learned  a  rule  and  the  subjects 
were  coded  as  doing  regular  e.xplanation.  There  are  two  possible  ex¬ 
planations  of  this  discrepancy.  Either  the  subjects  really  were  doing 
impasse-driven  learning,  but  they  showed  no  signs  of  it  in  the  protocol, 
or  the  subjects  knew  tiie  rule  already  and  were  simply  applying  it  here 
rather  than  learning  it.  Since  the  data  are  consistent  with  both  explana- 


tioiis.  the  iiilprpretntioii  depeiuls  on  tlie  prior  prol)al)ilitios  of  possessing 
llie  rules  in  (piestion.  All  bill  one  of  the  ten  rules  were  not  mentioned  in 
the  textbook,  so  they  are  less  likely  to  be  in  the  subjects'  initial  knowl- 
edfie.  The  rule  that  is  mentioned  in  the  textbook  determines  the  sign  of 
a  projection  of  a  vector  onto  a  negative  axis.  However,  three  subjects 
showed  clear  signs  of  impasses  when  this  rule  was  first  used,  and  three 
used  a  buggy  version  of  the  rule  that  always  assigned  a  positive  sign. 
The  correct  sign  rule  appears  to  be  hard  to  learn  and/or  recall  from  the 
text  for  ()  of  the  9  subjects,  so  it  was  probably  not  known  by  the  other 
subjects  either.  Thus,  because  none  of  the  rules  involved  seem  likely  to 
Ix'  in  the  subjects'  initial  knowledge,  we  suspect  that  all  It)  cases  in  this 
cf'll  of  Table  ')  correspond  to  impasse-driven  learning  events  that  were 
not  displayed  by  the  subjects. 

I  here  were  4  cases  where  the  subject  clearly  tried  to  explain  a  goal 
but  failed.  On  subsecpient  problems,  the  subject  would  explicitly  refer 
back  to  these  iioints  in  the  examples  and  use  transformational  analogy. 
This  is  just  what  the  two  subjects  who  were  coded  as  doing  analogical 
abduction  do.  so  that  is  why  we  now  believe  that  there  is  no  evidence 
for  analogical  abduction. 

There  are  49  cases  where  Cascade  did  regular  explanation  and  the 
subjects  were  coded  as  doing  accepts.  Of  these.  35  occurred  when  Cas- 
( ade  was  frying  to  explain  a  force  diagram.  .According  to  its  rules,  figur¬ 
ing  out  whi(  h  forces  exist  and  determining  their  directions  are  snbgoals 
deeply  embedded  beneath  the  goal  of  drawing  the  force  diagram.  Some 
subjects  discussed  the  forces  without  mentioning  the  force  diagram.  Ei¬ 
ther  they  were  silently  explaining  many  of  the  details  of  the  force  dia¬ 
gram.  or  they  ignored  the  force  diagram  and  “just  knew"  that  it  was 
im|)orlanl  to  exjilain  the  forces.  We  think  the  latter  is  more  plausible, 
but  we  cannot  easily  model  it  without  changing  the  goal  structure  em¬ 
bedded  in  ( 'ascade's  rules  or  Cascade's  model  of  accepting  the  exami)le's 
stali'inenls  without  irroof. 

Stepping  back  from  the  details,  there  are  several  results  from  the 
analysis  in  Table  5.  It  is  clear  now  that  Cascade  needs  several  kinds  of 
revisions:  (1)  Analogical  abduction  should  be  eliminated  and  transfor¬ 
mational  analogy  strengthened.  (2)  The  goal  structure  and/or  the  ac- 
re])iance  mechanism  need  to  be  revised  in  order  to  handle  some  subjects' 
cxpianaf ions  of  free-body  diagrams.  (3)  The  subjects  showed  impasse- 
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like  behavior  on  -1  occasions,  but  seemed  to  learn  nothing  from  them. 
1  his  is  rnirently  not  an  ojilion  willi  (‘asrade.  It  should  be  rlianged  so 
I  ha'  one  of  its  responses  to  an  impasse  is  just  to  accept  the  stuck  goal 
as  true  without  learning  anything. 

.Another  result  concerns  the  handling  of  impa.sses.  Cascade  had  19 
im|>asses  where  learning  occurred,  and  of  these,  the  subjects  showed 
im|>asse-like  Irehavior  on  9.  If  we  believe  the  codings,  then  48%  of  the 
sul)j('ct's  impasses  where  learning  occurred  were  visible  in  protocol  data, 
and  I  he  other  52%  were  the  victim  of  the  usual  incompleteness  of  protocol 
data.  In  the'  X'anLehn  (1991a)  study  of  imjrasses  in  strategy  discovery, 
tlierc'  were  1 1  learniitg  event  s,  of  which  8  (73%)  were  marked  Iry  impasse¬ 
like  behavior  on  the  sulcject's  part.  This  is  consistent  with  the  impression 
that  one  gets  from  reading  the  protocols,  which  is  that  the  snieject  in  the 
X'anLehn  (1991a)  study  verbalizes  much  more  of  her  thoughts  than  the 
subjects  in  the  Chi  et  al.  (1989)  study.  Thus,  although  the  proportion 
of  "silent"  im])asses  is  higher  in  the  present  study,  it  is  not  inconsistent 
with  the  earlier  study's  proposition. 

How  many  of  the  subjects’  explanations  are  matched? 

The  preceding  section  evaluated  the  match  in  one  direction  only,  by 
seeing  how  mndi  of  Cascade's  behavior  is  matched  lyv  subject  behavior. 
This  section  evaluates  how  much  of  the  sulrject's  behavior  is  matched  by 
( 'nscade  behavior. 

In  order  to  do  this,  we  e.xtended  an  analysis  by  Chi  and  VanLehn 
(1991  ).  They  first  coded  every  utterance  in  the  example  studying  pro¬ 
tocols  as  either  a  physics  explanation,  a  mathematical  explanation  or 
one  of  several  other  kinds  of  utterances.  They  then  co<led  each  ])hysics 
explanation  into  an  if-then  rule  that  presented  the  gist  of  the  subject's 
comment  in  a  uniform,  more  easily  understood  format.  We  extended 
this  analysis  Iry  including  the  mathematical  explanations  as  well  and  Iry 
correcting  what  we  felt  were  a  few  minor  mistakes  in  the  earlier  analysis 
of  physics  explanations.'’  Finally,  we  determined  whether  each  rule  ap- 
jrarently  used  by  the  stibjects  was  also  contained  in  Cascade's  knowledge. 

'’It  was  usually  clear  what  the  action  sides  of  these  rules  should  be.  but  inferrinji;  the 
preconditions  and  (he  generality  of  the  rules  often  required  tnakiiiR  some  assumptions, 
‘'ono  i inies  we  disanreeil  with  the  assumptions  made  in  the  earlier  analysis. 
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T;)l)l<‘  (i:  Sul)jorfs  explanations  dnring  example  studying 


Cateaories 

143 

Matches  Cascade  (63%) 

61 

Outside  task  domain  (27%) 

2  3  Mathematical  manipulations 

9  Editting  part  a  to  solve  part  b 

1  6  Extra  example  lines 

1  3  Units  or  terminoloov 

6 1  Total 

23 

inside  task  domain  (10%) 

8  Incorrect  explanations,  retracted 
6  Acceleration  and  motion 

3  Abstract,  partial  plans 

0  Global  planning  from  Chi  et  al. 

6  Miscellaneous.  ooDortunistic 

23  Total 

227 

Grand  total 

Some  of  this  knowledge  ap])eared  explicitly  as  Cascade  rules,  while  some 
of  it  was  implicit  in  Cascade's  rule  interpreter  (e.g..  algebraic  knowledge). 

Of  the  227  total  explanation  episodes  found  in  the  protocols,  we  de¬ 
termined  that  l-i:}  (G.3%)  were  matched  by  Cascade  explanations,  while 
S-l  were  not.  This  indicates  that  Cascade  models  a  large  portion  of  the 
siib  jocts*  explanatory  behavior.  However,  there  are  also  many  explana¬ 
tions  that  Cascade  does  not  appear  to  account  for.  We  categorized  these 
explanations  in  order  to  determine  whether  Cascade  should  be  expected 
to  make  them  (see  Table  (i). 

Of  the  N4  explanations.  61  concerned  reasoning  that  was  outside  the 
domain  of  cognition  being  modeled.  There  were  four  cla.ssifications: 

•  2'f  explanations  concerned  mathematical  inferences  that  Cascade 
did  not  need  to  make  because  it  had  either  been  gi%en  the  informa¬ 
tion  in  the  problem  statement  (e.g.,  certain  geometric  information 
was  provided  to  it),  or  it  did  not  simplify  its  answers. 

•  9  explanations  occurred  exclusively  on  part  6  of  one  example.  Part 
a  of  I  his  example  describes  a  static  situation  where  a  string  is  hold- 
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inp,  a  l)lo(  k  on  an  inclined  plane.  Part  b  of  the  example  ask.s  one 
to  find  I  lie  arrelorntinn  of  the  hh^rk  when  the  string  is  rnt.  Snli- 
je<  ts  explained  part  b  hy  “editing"  the  explanation  of  i)art  a.  For 
instance,  one  editing  rule  was  “If  a  force  is  removed  from  a  static 
case  and  there  is  no  friction,  the  body  will  move."  Cascade  does 
not  reason  about  part  b  of  the  example  in  this  manner.  Rather, 
the  system  treats  jrarts  a  and  6  as  two  distinct  exami)les. 

•  Sometimes  an  example  would  contain  a  few  lines  that  would  em- 
])hasize  aspects  of  the  problem  that  were  not  germane  to  solving  it 
or  would  discuss  limiting  ca.ses  (see  Figure  1.  line  3).  Although  we 
did  not  ask  Cascade  to  explain  these  lines,  the  sulijects  sometimes 
would,  and  Iti  of  their  explanations  were  of  this  type. 

•  13  explanations  involved  miscellaneous  comments  about  the  exani- 
)>les  that  were  judged  to  represent  knowledge  outside  of  the  task 
domain  as  formalized  in  Cascade.  For  in.stance.  we  did  not  bother 
to  model  reasoning  with  units,  so  the  statement.  “In  the  English 
system,  slugs  are  mass  and  pounds  are  weight."  is  considered  out- 
sid('  the  task  domain. 

File  remaining  23  exiilanatioiis  are  all  relevant  to  the  domain  modeled 
by  ( 'ascade.  so  it  should  [irobably  generate  them.  They  fell  into  three 
classificat  ions. 

•  K  explanations  appear  to  have  been  generated  tentatively  then  re¬ 
tracted.  They  are  all  incorrect  statements  about  physics,  and  the 
sub  jects  seem  to  have  revised  their  explanation  a  short  time  later. 
.\n  example  is.  "If  the  two  bodies  in  an  .Atwood's  machine  have  the 
same  acceleration,  then  they  are  not  moving."  How  the  subjects 
generated  these  conclusiotis  is  a  bit  of  a  mystery,  although  some  are 
cletirly  the  results  of  overly  general  rules.  For  instance,  one  subject 
said  that  one  can  calculate  the  tension  in  a  string  by  adding  the 
tensions  in  its  parts.  He  probably  generated  this  explanation  by 
applying  an  overly  general  part-whole  rule  that  works  correctly  for 
quantities  such  as  volume,  mass  and  weight. 

•  (')  explanations  concerned  the  relationship  between  motion  and  ac- 
(flcraiion.  Of  these,  -1  explanations  stated  essentially  the  same 
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lliiiiR:  “If  a  body  moves.  Uieii  it  lias  a  uon-zero  acceleralion."  All¬ 
ot  tier  ex|)lniiatioii  sniil.  “If  a  lioily  lias  motion  on  an  axis,  then  it 
has  acceleration  on  that  axis."  and  the  last  explanation  said.  “If 
acceleration  is  zero,  then  nothing  is  moving.”  These  all  stem  from 
the  same  incorrect  conception  of  acceleration  as  speed,  which  is 
very  common  and  hard  to  remove  (Reif.  19^7).  Cascade  should 
also  lie  e(|iiip])ed  with  this  misconception.  Even  though  these  new 
rules  would  alter  Cascade's  model  of  the  subjects'  explanation  of 
examples,  we  do  not  expect  that  they  would  change  Cascade's  be¬ 
havior  on  later  problems.  This  is  because  the  problems  all  deal  with 
acceleration  and  do  not  mention  concepts  like  velocity  or  “motion.'' 

•  .'5  ex|)lanations  articulated  an  abstract,  partial  plan  for  solving  the 
|)rol)lem.  For  instance,  one  explanation  said  “The  weight  of  the 
lilock  in  the  string  example  can  be  computed  by  figuring  out  the 
tension  in  the  strings.”  This  rule  represents  the  toj)  level  of  an 
abstract  plan  for  determining  the  weight  of  the  block.  Cascade 
does  tiot  do  hierarchical  planning,  but  this  rule  provides  evidence 
that  perhaps  it  should.  In  particular,  we  would  probably  find  more 
evidence  for  planning  in  the  problem  solving  protocols.  Planning 
did  not  have  much  of  a  chance  to  emerge  during  example  studying 
liecaiise  sulijects  are  mostly  led  by  the  hand  through  the  example 
solutions,  so  there  is  no  real  need  to  plan. 

•  <i  explanations  defy  classification,  so  they  are  simply  listed  below 

1.  The  body  is  the  thing  that  the  forces  are  acting  oti. 

2.  Tetision  is  importatit  because  it  transmits  the  force  between 
t  he  blocks. 

d.  The  acceleration  in  ati  .Atwood's  machine  is  caused  by  gravity. 

1.  If  the  right  mass  is  greater  than  the  left  mass  in  an  .Atwood's 
machine,  then  the  machine  will  accelerate  downward. 

5.  .An  upward  force  can  act  against  gravity  to  keep  a  body  from 
falling  down. 

I).  .Most  forces  are  gravitational. 
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All  ()  explanations  are  true  statements.  However,  they  are  not 
n'levaiil  to  the  Roal  of  solving  the  problem,  which  is  why  Cascade 
did  not  make  them. 

This  analysis  indicates  that  of  the  227  explanations  uttered  by  sub¬ 
jet  is.  1  |.{  (tid'd  )  were  matched  by  Cascade's  explanations.  61  (21%)  were 
outside  the  task  domain  Ireing  modeled,  and  23  (lO'X  )  are  explanations 
that  ( 'ascade  siionld  make  but  does  not. 

Cascade  embodies  a  hypothesis  about  explanation,  which  is  that  ex¬ 
planation  of  solution  lines  in  physics  examples  consists  of  rederiving 
them.  This  is  a  kind  of  local  explanation,  in  that  Cascade  focuses  only 
on  the  current  solution  line.  It  does  not  step  back  and  try  to  see  a 
gloiial  pattern  that  spans  all  the  solution  lines  in  an  e.xample.  This  may 
seem  somewhat  unusual,  as  other  models  of  example  explaining  (e.g.. 
\auLe|in.  1!)')0:  Reimann.  in  jiress )  emphasize  glolral  explanation.  How¬ 
ever.  from  one  point  of  view,  there  is  little  point  in  global  explanation  of 
physic  s  example's.  Because  later  solution  lines  use  results  from  earlier  so¬ 
lution  line's,  doing  local  explanation  of  the  later  lines  ties  them  together 
with  the  earlic'i  line's,  yielding  an  overall  coherent  structure.  From  an¬ 
other  point  of  vic'w.  there  is  great  benefit  in  glolral  explanation,  because 
it  turns  out  that  all  theexamjeles  have  a  similar  chronological  structure: 
one  first  choosc's  bodies,  then  draws  a  diagram  showing  all  the  forces 
IK  tiuii  on  the  bodies,  chooses  coordinate  a.xes.  instantiates  Newton's  law 
aloim  c'ach  axis,  attd  solves  the  resulting  system  of  equations.  The  text- 
l)ook  even  tiientions  this  procedure.  One  might  expect  the  subjects  to 
look  for  such  a  global,  chronological  structure,  perhaps  by  first  locally 
explaining  all  the  solution  lines,  then  refiecting  on  the  whole  solution  to 
sc'c'  if  ill*'  overall  stntcture  made  sense.  However,  the  subjects  produced 
oul\  3  global  explanation  statements.  Seeing  the  overall  chronological 
pattern  in  solutions  does  not  appear  to  be  a  major  concern  for  these  sub¬ 
jects.  ])erhaps  because  the  logical  structure  suffices  to  make  lines  cohere, 
riiis  is  coiisisteiii  with  Sweller's  work,  which  has  shown  that  chronolog¬ 
ical  patfertis  eiiit)edded  iti  solttfioits  are  often  overlooked  when  sitbjects 
ate  fo<  tissed  ott  tibtaitiittg  a  goal  (Swellet  A  Levine.  16'<2). 

.Attother  hypothesis  about  explaitatioti  entlrodied  in  Cascade  is  that  all 
ittferetices  are  ultimately  directed  towards  the  top  level  goal  of  explaining 
the  citrreut  sobttiott  line.  Cascade  uses  a  backwards  chaining  theorem 
piovei.  whi(h  meatis  that  it  starts  with  the  top  level  goal,  chooses  and 
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ai)i>li(“;  a  nilo.  ntui  tlu'ii  fo(‘ust*s  on  tlie  first  of  the  several  subgoals  cre- 
ai('<!  b\  lli('  rule's  a|)i»liralion.  W’lioii  all  sulrgoals  have  been  arliieved.  it 
executes  the  rule,  whieh  finally  draws  a  conclusion.  This  means  that  all 
inferences  are  done  in  order  to  satisfy  some  goal,  and  that  goal  is  ulti¬ 
mately  a  subgoal  of  the  toj)  level  goal.  This  makes  Cascade  a  narrowly 
focused,  methodical  explainer.  It  could  be  that  people  are  more  opiror- 
tunisiistic  and  make  observations  (i.e..  drawing  conclusions)  whenever 
thev  notice  that  they  can  be  drawn.  In  the  e.xtreme.  they  might  do  for¬ 
ward  chaining,  drawing  all  irossible  conclusions  about  a  problem  while 
par  ing  no  attention  whatsoever  to  the  solution  lines  or  the  overall  goal 
of  the  problem.  However,  only  (i  of  the  '2‘27  explanations  apjrear  to  be 
o|>port unist ic.  in  that  they  were  not  nratched  by  Cascade's  goal-directed 
inferences. 


How  much  of  Cascade’s  problem  solving  is  matched? 

Ue  turn  now  to  considering  problem  solving  behaviors.  The  com- 
paiisoii  bc'tween  Cascade  and  the  subjects  is  made  difficult  by  two  fac¬ 
tors.  f  irst,  the  |)rotocols  are  huge.  There  are  approximately  2700  pages 
of  problem  solving  protocol,  as  compared  to  300  pages  of  the  example 
studviiig  protocol.  Second,  problem  solving  is  less  constrained  than  ex- 
atnple  studying.  Rc'deriving  a  line  in  a  solution  takes  at  the  very  most 
oid\  a  few  minutes,  whereas  solving  a  problem  can  take  almost  an  hour. 
Subjet  ts  seldom  get  badly  lost  while  rederiving  a  solution  line,  whereas 
w  hen  solving  a  problem,  subjects  often  wander  down  several  unproduc¬ 
tive  ])aths  before  finding  a  solution  or  giving  up.  Getting  Cascade  to 
follow  the  sitbjects  on  a  long  doomed  search  path  can  be  difficult. 

,\n  important  methodological  problem  is  finding  a  fair  way  to  evaluate 
the  fit  of  a  simulation  and  a  protocol.  .Sui)i»o.se  one  evaluated  the  fit 
by  ((Uniting  the  actions  taken  by  the  simulation  that  are  matched  by 
subject  actions,  and  dividing  by  the  total  number  of  actions  taken  by 
the  simulation.  It  often  happens  that  the  actions  of  the  simulation  and 
the  subject  disagree  at  some  point.  This  often  causes  them  to  diverge 
and  follow  se|)arate  paths  for  a  wliile.  perhaps  even  a  long  while.  The 
longer  the  divergent  paths,  the  worse  the  fit.  even  though  the  blame 
is  due  to  one  false  move  earlier  in  all  cases.  Thus,  simply  comparing 
maidiing  to  mismatching  actions  is  unilluminating,  for  it  confounds  the 


(inulily  of  llio  siinulnlion  with  i)ioj)erties  of  the  search  space,  namely,  the 
h’liRlhs  of  reiiaiii  ]>nths. 

Our  approach  is  toecpiip  Cascade  with  a  variety  of  parameters,  which 
we  call  ‘•nudges."  whose  main  pnr])ose  is  to  midge  Cascade  back  onto  the 
subject's  search  path  whenever  it  would  otherwise  wander  off.  The  fit 
between  the  simulation  and  the  subject's  protocol  is  measured  by  count¬ 
ing  the  number  of  times  the  simulation  had  to  be  nudged.  .4n  adriitional 
benefit  of  nudging  for  measuring  fit  is  that  each  nudge  represents  a  piece 
of  unexplained  cognition.  By  taking  a  census  of  the  nudges,  one  can 
rank  types  of  unmodeled  cognition  and  discover  which  ones  are  affecting 
problem  solving  behavior  the  most.  Here  are  the  ty])es  of  nudges  used: 

•  When  ( 'ascaile  solves  a  problem,  it  normally  tries  transformational 
analogy  only  after  it  tries  regular  domain  knowledge.  However, 
some  subjects  apiiarently  prefer  to  use  transformational  analogy  in 
some  cases  even  when  their  behavior  on  earlier  cases  demonstrates 
that  they  have  the  appropriate  domain  knowledge  and  could  pulen- 
tially  use  it  here.  In  order  to  force  Cascade  to  follow  the  subjects, 
pioposilioiis  of  the  form  traf o-only (G)  were  placed  in  the  jrrob- 
lem's  description  whenever  G  is  a  goal  that  the  sniiject  preferred  to 
adiieve  via  transformational  analogy.  We  call  such  cases  of  trans¬ 
formational  analogy  "forced.'' 

•  Mv  default,  the  toji-level  goals  of  a  problem  reiiuire  identifying  a 
body  and  drawing  a  free-body  diagram  before  finding  the  sought 
(|nantities.  If  the  subject  did  not  draw  a  free-body  diagram,  we 
eliminated  these  goals  from  the  statement  of  the  problem. 

•  Oti  rare  occasions,  subjects  came  up  with  analogical  mappings  that 
(  a'cade  coidd  not  generate.  In  such  cases,  we  simply  gave  (  ascade 
the  subject's  ma])ping  or  modified  the  problem  representations  so 
that  the  subject's  mappings  could  be  generated. 

•  Subjects  (lid  not  always  use  the  buggy  F  =  ntn  rule,  which  lets  F  be 
an  individual  force  instead  of  the  net  force,  when  it  was  applicable 
(i.e..  we  could  not  figure  out  exactly  what  preconditions  the  sub¬ 
jects  had  for  this  rule).  Perhaps  they  rightly  believed  that  it  wa.s 
overly  general,  and  thus  they  would  only  use  it  as  a  last  re.sort.  .^t 


any  rate,  we  controlled  its  usage  by  entering  ignore (F=ma_wrong) 
into  the  descriptions  of  some  problems  but  not  others. 

In  order  to  evaluate  Cascade's  ability  to  model  a  given  subject,  nudges 
were  enteretl  by  trial  and  error.  Cascade's  behavior  then  was  compared 
to  the  snl)ject's  by  classifying  each  of  the  goals  in  its  trace  according  to 
Innv  the  goal  was  achieved.  Four  classifications  were  used: 

•  Regular  solving:  Cascade  used  one  of  the  domain  rules. 

•  Impasse  with  learning:  Cascade  reached  an  impasse,  successfully 
applied  EBLC.  and  learned  a  new  domain  rule. 

•  Iratisformational  analogy:  Cascade  could  achieve  the  goal  with  a 
domain  nde.  so  it  used  transformational  analogy. 

•  F'orced  transformational  analogy:  W’e  forced  Cascade  to  use  trans- 
ft)rmational  analogy  even  though  it  could  have  used  a  domain  rule 
to  achieve  the  goal. 

Nb'xt.  each  of  these  goals  was  classified  according  to  how  the  subject 
appeared  to  achieve  it.  Four  classifications  were  used  here  as  well: 

•  Iransformational  atialogy.  If  the  siibject  referred  to  an  example 
and  copied  parts  of  it  over,  the  goal  was  classified  as  achieved  by 
t ransformat iotial  analogy. 

•  Ititpasse  with  learning.  If  the  subject  paused,  complained  abottt 
the  goal  or  iti  some  other  way  showed  signs  of  being  stuck,  but  the 
subject  did  not  refer  to  an  example  to  achieve  the  goal,  then  we 
( lassified  the  goal  as  being  resolved  by  EBLC. 

•  Impasse  and  give  up.  On  a  few  impasses,  the  subject  just  gave  up 
w  if  hont  seeming  to  lesohe  the  impasse  or  learn  an>’  new  rules.  Civ- 
itig  up  is  lurt  one  of  the  options  availalrle  to  Cascade  for  handling 
an  impasse,  although  it  should  l)e. 

•  Regular  solving:  If  the  subject  solved  the  goal  without  complain¬ 
ing.  referring  to  an  example  or  pausing  for  inordinate  amounts  of 
time,  then  we  classified  the  subject  as  achieving  the  goal  via  regular 
|)robh>m  sobing. 


Tiil)lc  7:  How  many  Cascade  goals  are  achieved  the  same  way  by  sub¬ 
jects? 


Caccada 

tubloeia  I 

Totals 

Ragutar  aolving 

47 

0 

0 

3700 

Tranalerni.  analogy 

176 

0 

4 

106 

Fofcad  irana.  analogy 

35 

0 

0 

35 

ImoBssa  wHh  laarnma 

0 

« 

0 

16 

Totals 

3676 

258 

0 

4 

3047 

Civen  tliese  classifications,  the  results  for  all  nine  subjects  appear  in 
Table  7.  In  most  cases  (dG'j3  +  176  +  9  =  3S3S  or  97%),  the  subjects 
handled  goals  in  the  same  way  that  Cascade  did.  which  was  extremely 
unlikely  to  ocoir  by  chance  {p  <<  .001.  Chi-squared  test).  Let  us  ex¬ 
amine  each  of  the  other  ceils  to  see  how  serious  the  mismatching  ca.ses 
are. 

There  were  17  cases  where  the  subjects  did  tran.sformatioual  analogy 
and  Cascade  did  not.  All  these  were  due  to  the  simplicity  of  Cascade's 
nuxlel  of  transformational  analogy.  One  subject  often  used  vector  equa- 
1  ions  as  if  I  hey  were  scalar  ecpiaf  ions,  and  applied  them  in  creative,  albeit 
incorrect  ways  that  Cascade  could  not  model.  VVe  let  Cascade  solve  those 
l)rol)lems  in  its  normal  way  and  counted  all  12  goals  as  ca.ses  where  the 
sub  ject  did  t ransformat iotial  analogy  and  Cascade  did  not.  The  other  5 
ca.ses  occurred  when  a  subject  could  not  recall  some  trigonometry  rules, 
so  she  mixed  transformational  analogy  with  regular  problem  solving  in 
a  complex  way  that  Cascade  could  not  mo<lel.  These  17  ca.ses  indicate 
that  (.'ascade's  model  of  transformational  analogy  needs  improvement. 

There  were  16  cases  where  Cascade  did  transformational  analogy  and 
the  subject  seemed  to  do  regular  problem  solving.  Transformational 
analogies  occur  when  Cascade  is  missing  the  knowledge  to  do  regular 
problem  solving,  so  there  are  two  possible  explanations  for  each  case: 
Either  the  subject  knew  the  rules  that  Cascade  lacked,  or  the  subject 
actually  did  have  an  impasse  and  resolved  it  with  transformational  anal¬ 
ogy.  but  (hey  didn't  refer  overtly  to  the  example  because  they  could 
remember  the  line  that  they  needed,  and  thus  were  not  coded  as  j)er- 
forniing  transformational  analogy.  Of  the  16  cases,  8  seemed  to  be  cases 
of  covert  transformational  analogy  because  they  involve  accessing  the 
fi»'p-l)odv  diagram,  which  is  much  easier  to  remember  than  the  equa- 
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(ions.  Two  more  cases  seemed  to  be  covert  transformational  analogy. 
Imcanso  the  siil)je<l  hail  already  referred  to  the  example's  ecpialion  d 
limes  earlier,  and  probably  had  committed  it  to  memory.  Five  cases 
seemed  to  be  caused  by  the  subject  having  a  buggy  rule  about  negative 
signs  that  Cascade  lacked,  but  Cascade  was  able  to  get  the  same  eflect 
with  transformational  analogies.  The  last  case  is  similar  to  the  five  just 
(lisi  ussed.  but  with  a  different  rule.  Thus,  of  the  16  cases.  10  seem  to 
be  (overt  impasses  correctly  predicted  by  Cascade,  and  6  seem  to  re¬ 
sult  from  the  subject  having  initial  knowledge  that  is  not  in  Cascade's 
standard  initial  knowledge  ba.se. 

In  1  cases,  the  subjects  reached  impasses  and  gave  up.  Since  Cascade 
curri'iitly  cannot  give  up  at  impasses,  these  cases  were  approximated  with 
transformational  analogy.  However.  Cascade  did  succeed  in  predicting 
the  location  (»f  the  impasses. 

I'here  were  cases  of  forced  transformational  analogy.  Two  subjects 
(0  of;!.')  cases)  always  copied  the  free- body  diagrams  and  never  generated 
them  on  their  own.  so  these  subjects  apparently  were  lacking  knowledge 
about  drawing  free-body  diagrams.  In  retros|)ect.  these  subjects  should 
have  lieen  modeled  by  having  their  initial  knowledge  adjusted  to  remove 
the  rules  about  drawing  free-body  diagrams.  The  other  '26  cases  occurred 
with  subjects  who  clearly  had  the  rerpiisite  rules,  but  chose  to  do  trans¬ 
formational  analogy  instead.  Of  these  26  ca.ses.  21  involved  cojtying  a 
free-body  diagram  rather  than  reasoning  it  out  from  physics  princi))les 
and  the  other  •')  involved  cojning  trigonometry  functions  rather  than  fig¬ 
uring  out  whether  the  function  should  be  sine,  cosine  or  tangent,  and 
what  the  angle  should  l)e.  It  is  certainly  easier  to  use  transformational 
analogy  for  these  particular  ca.ses.  atid  apparently  the  subjects  felt  it  was 
safe  to  do  so.  even  though  transformational  analogy  is  fallible. 

riiei'e  w('re  7  cases  where  Cascade  did  impasse-driven  learning  and 
the  subjects  were  coded  as  doing  regular  problem  solving  because  they 
showed  no  signs  of  impasses.  In  general,  there  are  two  possible  expla¬ 
nations  for  such  cases.  Either  the  subject  actually  did  impasse-driven 
learning  Imt  failed  to  show  any  signs  of  it  in  their  protocol,  or  they 
already  had  the  rule  that  Cascade  was  missing  so  they  just  applied  it 
instead  of  learning  it.  VVe  examined  each  of  the  7  cases  to  try  and  deter¬ 
mine  which  explanation  was  most  plausible  for  each.  In  some  problems 
with  friction  in  them.  Cascade  must  learn  four  rules  about  friction  forces. 


OiH'  siibjoci  showed  no  signs  of  an  impasse  at  any  of  these  locations,  so 
we  siisi)erl  that  she  already  understood  friction  forces.  Another  subject 
showed  signs  of  an  impasse  at  three  of  these  places,  but  not  at  the  fourth; 
it  is  likely  that  this  fourth  occasion  was  a  covert  impasse.  Similarly,  one 
problem  rerpiired  learning  four  rules  about  pressure  forces.  A  subject 
showi'd  sijins  of  an  impasse  on  only  two  of  the  four  occasions,  so  it  is 
likely  that  the  other  two  occasions  are  silent  impasses.  Thus,  of  the  7 
cases  where  Cascade  does  impasse-driven  learning  and  the  subjects  ap¬ 
pear  not  to.  d  seem  to  be  silent  cases  of  impasse-driven  learning  and  4 
seem  to  be  cases  where  the  subject  already  knew  the  rules  that  Cascade 
learned. 

From  this  e.xnmination  of  the  mismatching  cases,  it  seems  that  Cas¬ 
cade  woitld  need  three  augmentations  in  order  to  handle  all  the  data.  ( 1) 
1  ianslormat ional  analogy  needs  to  be  made  more  powerful  so  that  it  can 
model  the  more  creative  (albeit  incorrect)  usages  e.xhibited  by  subjects. 
(J)  I  he  model  should  be  free  to  choose  transformational  analogy  instead 
of  regular  ))roblem  solving  when  it  estimates  that  transformational  anal¬ 
ogy  would  be  easier  or  more  reliable  than  regular  problem  solving.  (3) 
When  Cascade  cannot  resolve  ati  impasse,  it  should  be  allowed  to  give 
up.  .All  the  other  cases  of  mismatching  appear  to  be  covert  versions 
of  the  predicted  events,  or  cases  where  rules  should  have  been  removed 
from  the  initial  knowledge  base. 

In  short,  it  appears  that  almost  everything  that  Cascade  does  is 
niatdied  bv  snbjeit  behavior.  The  next  section  analyzes  the  subjects' 
Itehavior  in  order  to  see  how  much  of  it  is  matched  by  Cascade. 

How  much  of  the  subjects’  problem  solving  behavior  is 
matched? 

In  order  to  ((iianlifv  how  much  of  the  subject's  thinking  during  prob¬ 
lem  solving  could  be  simulated  Ity  Cascaile.  we  adojrted  the  same  unit 
of  an;il\sis  that  was  used  in  the  preceding  analyses  by  converting  the 
subjects'  protocols  intcc  Cascade-sized  goals.  .As  an  ilbistration  of  this 
analysis,  .\ppendix  1  shows  a  protocol  and  our  encoding  of  it.  Following 
the  tradition  of  .Newell  and  Simon  (1972),  the  protocol  appears  in  the 
left  column,  and  the  encoding  appears  in  the  right  column. 

This  kitid  of  analysis  is  (ptite  time  consuming,  so  we  could  not  do  it 
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t(’r  all  2')2  |)iolo(ols.  TIuis.  we  selected  4  protocols  tliat  we  felt  were 
ty|)i(  al.  4' wo  wore  from  CJood  solvers,  and  two  were  from  Poor  solvers. 
Lacli  of  these  pairs  consisted  of  one  protocol  that  was  mostly  transfor¬ 
mational  analogy  and  one  that  was  mostly  regular  rule-based  reasoning. 
(  The  protocol  in  .-Vppendi.x  I  is  a  Poor  solver  who  is  doing  mostly  trans¬ 
formational  analogy.)  (  'learl>.  this  is  too  small  a  sample  to  draw  strong 
infenMices.  hut  out  point  in  this  section  is  just  to  get  a  rough  idea  of  the 
mate  h. 

Inferences  occur  whenever  a  goal  is  reduced  to  subgoals,  or  a  goal 
is  achieved.  Thus,  by  literally  reading  between  the  lines,  one  ran  tell 
from  the  encoded  protocols  what  the  subjects'  inferences  were.  In  the 
four  protocols,  there  were  l.'jl  inferences,  e.xcludlng  trivial  arithmetic  and 
algc'braic  onc's.  \\V  examined  each,  and  determined  that  Cascade  could 
do  all  but  1')  of  them.  That  is.  if  we  were  to  simnlate  these  protcjcols 
with  <'ascacb“.  we  would  need  lo  new  rules  and  would  probably  have  to 
nudge  it  I-')  times  in  older  to  gel  it  to  apply  these  rules.  Thus,  it  appears 
that  Cascade  can  model  alroiit  90/T  of  the  subject's  inferences  during 
problc'in  sohing.' 

In  orcIcM'  to  give  a  cpialitat  ive  sense  of  the  behavior  that  (.'ascacle  ct>ulcl 
not  simitlalc’.  we  divided  the  inferences  into  ones  that  seemed  outside 
of  the  intended  task  domain  of  Cascade  and  those  that  Cascade  really 
should  hiive  modeled.  Those  that  are  outside  the  task  domain  are: 

•  In  2  itiferences.  the  subjects  checked  their  work  by  itlugging  their 
answers  back  into  ecpiations  and  seeing  if  the  e<|uations  balanced. 

•  In  2  inferences,  the  subjects  struggled  to  find  the  appropriate  units 
for  t heir  calculations. 

•  In  2  itiferences.  (hc>  subject  had  difficulty  understanding  the  dia- 
giam  that  accompanied  the  problem  statement.  In  iiarticnlar.  it 
was  diflicidt  to  decide  whether  a  certain  line  stood  for  a  string  or 
not . 

The  inferences  that  were  inside  the  intended  domain  were: 

Franklv.  lliis  e’ctiiiiate  sefius  to  us  If  we  artuallv  tried  to  add  the  rerinisite 

IS  rules  In  C''a.scade  and  siinidale  these  protocols,  we  would  probably  find  that  the 
coxerayf-  was  closer  to  fitter  or  T0‘7c. 


•  4  iiifpieiues  were  coded  for  a  case  where  the  subject  derided  he 
need('d  to  understand  freefall  better,  and  went  off  to  read  tlie  rele¬ 
vant  |)ap,e  of  t  he  text  book,  tlien  decided  that  his  (incorrect )  solution 
to  the  problem  was  right  anyway. 

•  2  inferences  were  coded  for  a  case  where  the  subject  decided  to 
convert  a  vertical  acceleration  to  one  parallel  to  an  inclined  plane, 
l)ut  apparently  did  not  realize  that  projection  could  be  applied 
directly,  so  he  •‘converted”  it  to  a  force  using  F  =  mu,  projected 
the  force,  then  converted  it  back. 

•  2  inferences  involved  a  subject  who  let  g  =  -9.8  for  no  apjrareiit 
reason.  This  could  have  been  an  unintentional  error,  except  that 
the  subjf'ct  noticed  later  that  g  was  negative  and  did  not  correct  it. 
riio  second  inference  occurred  later,  when  she  dropped  a  negative 
sign  "for  fiin"  as  she  put  it,  thus  effectively  canceling  her  earlier 
error  and  oirtaining  a  correct  answer. 

•  1  inference  was  coded  for  a  subject  who  invented  something  he 
called  a  "douljle  force"  that  included  both  gravitational  and  fric- 
I ioiial  influences. 

.\lt hough  these  lists  would  clearly  be  much  longer  if  more  protocols  had 
b('<‘n  analyzed,  and  the  small  sample  makes  any  statistical  inferences 
unsound,  taking  the  data  at  face  value  indicates  that  about  a  third  of  the 
unmatched  behavior  is  outside  the  task  domain,  and  the  other  two  thirtls 
is  behavior  that  Cascade  should  model.  Moreover,  most  of  the  behavior 
that  Cascade  shoidd  model  is  incorrect  reasoning  of  a  wide  variety  of 
types.  .More  research  is  needed  before  we  can  conclude  anything  about 
the  sources  of  t  liese  incorrect  inferences. 

Control  choices 

The  preredinu,  analyses  assessed  what  was  done,  but  ignored  the  order 
in  whidi  actions  took  place.  This  section  concerns  the  overall  control 
structure  a.s  well  as  the  local  choices  of  which  rule  to  try  first  in  achieving 
a  goal.  Both  these  factors  control  the  order  in  which  inferences  take 

place. 


39 


('asfjiilp  usps  a  backwards  chaining  control  structure.  A  goal  is  pro- 
c('ssp(l  l)\  splocting  a  rule,  then  posting  any  sul)goals  rp<pnre(l  by  that 
rnlc.  Afipi  the  sul)goals  have  been  achievetl.  the  rule's  conclusion  is  as¬ 
serted.  Thus,  a  goal  should  show  up  twice  in  a  protocol:  when  it  is 
first  posted  and  wheti  it  is  completed.  In  our  protocols,  sub  jects  did  not 
nsnally  talk  al)f)nt  their  goals  when  they  posted  them  (see  Aitpendix  1 ). 
altlmngh  they  often  mentioned  the  conclusions  that  were  made  when  a 
goal  was  achieved.  This  could  be  taken  as  a  sign  that  they  were  not 
following  a  backchaining  control  structure.  However,  a  control  .structure 
also  restricts  the  order  in  which  actions  can  take  place.  For  instance,  if 
.A  and  B  are  snbgoals  of  t’.  and  D  is  not.  then  the  order  A.D.B  cannot 
occur.  Thus,  the  ordering  iti  which  goals  are  achieved  is  diagnostic  of 
t  he  cont  rol  structure. 

.\s  ])art  of  the  analysis  in  the  preceding  section,  we  fit  a  backwards 
chaining  goal  strtictnre  to  the  subjects"  behavior  in  the  four  |)rotocols 
anal\  zed.  Of  the  approximately  151  goals,  there  were  three  cases  where 
backwards  chaiiutig  wottld  not  fit.  In  two.  the  subject  performed  goals 
prematurely  (i.e..  the  .A.D.B  ca.se  just  ntentioned).  During  the  third  case, 
the  sul>jecl  explicitly  decided  which  of  two  conjunctive  (sibling)  goals  to 
do  first.  Tliis  kind  of  search  control  occurs  in  some  means-ends  analvsis 
prul)Iem  solvers  (e.g..  Prodigy:  .Minton  et  al..  1989)  but  not  in  Casca  'e. 
iti  short .  t  he  availal)le  evidence  ittdicales  that  ( 'ascade's  cont  rol  st  rtict  u  ? 
is  not  a  l)ad  first  approximation  to  the  subjects' overall  approacli. 

The  only  search  control  decision  itiade  by  (.'ascade  is  which  rtile  to 
ap|)ly  given  that  more  than  one  matches  the  current  goal.  Two  factors 
(letertnitie  Cascade's  choice  of  rule.  If  analogical  search  cotttrol  can  find 
an  old  goal  that  is  isomorphic  to  the  current  goal,  then  the  old  goal's 
rtde  is  chosen.  If  analogical  search  control  does  not  apply.  Cascade  selects 
rules  ill  the  order  in  which  they  appear  in  a  file.  This  file  is  set  u))  to 
generate  eflicieiit  beliavior  in  general,  atid  is  not  tuned  for  atiy  particular 
sii  1)  ject . 

riie  first  analvsis  involves  jilacing  all  goals  in  one  of  two  classes;  Ei¬ 
ther  the  first  rule  selected  for  achieving  this  goal  was  ultimately  rejected 
and  another  rule  was  used  in  its  place  in  the  final  solution,  or  the  first 
rule  selected  was  used  in  the  final  solution.  This  categorization  was  car¬ 
ried  otit  for  all  Cascade  goals  and  for  all  subject  goals  corresponding  to 
Cascade  goals.  Of  the  3401  cases  where  Cascade  picked  the  correct  rule 
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first,  llio  stihject  failed  to  pirk  the  correct  rule  first  in  only  81  (2.3%) 
(  ases.  Thus.  (  asrade  predicted  the  subject's  choice  of  rule  in  almost  all 
(f)7.7‘/I  )  cases. 

In  order  to  determine  how  much  of  this  success  is  due  to  analogical 
search  control,  we  s])lit  the  34G1  cases  according  to  whether  Cascade's 
choice  was  detei iniiied  by  analogical  search  control.  Cascade  selected 
th('  correct  rule  first  xntlwiit  the  involvement  of  analogical  search  control 
2702  times.  In  ()2  of  these  cases,  the  subjects  did  not  choose  the  correct 
rule  first.  Thus  Cascade's  default  rule  ordering  predicted  the  subjects' 
rule  choices  07. 7%  of  the  time.  Cascade  picked  the  correct  rule  first  will) 
its  analogical  search  control  mechanism  7.'39  times,  and  agreeing  with  the 
siil)  ie(  t's  choice  in  all  but  19  cases,  for  a  success  rate  of  97. •'3%. 

Initially,  this  seemed  a  disturbing  result,  because  it  appeared  that 
analogical  search  control  gives  the  model  no  predictive  accuracy  over 
the  default  rule  ordering.  This  raises  the  question  of  whether  Cascade 
would  be  better  off  without  analogical  search  control.  If  it  always  used 
its  default  rule  choice,  would  its  overall  prediction  accuracy  rise?  Rather 
than  simulate  all  9  subjects  with  analogical  search  control  turned  off.  we 
estimated  what  the  fit  would  be.  \\e  gave  Cascade  all  the  knowledge 
lu'cessary  to  exi)lain  and  solve  the  e.xamples  and  problems  (so  no  im- 
pass('s  would  be  generated)  and  ran  it  on  all  the  examples  and  proltlems. 
While  nmiting.  it  kept  track  of  how  many  times  it  used  analogical  search 
control  to  choose  a  rule,  and  how  many  times  that  choice  corresponded 
to  the  default  rule  choice.  We  found  that  analogical  search  control  led  to 
a  ciioice  different  from  tiie  default  rule  appro.vimately  12VT  of  the  time. 
This  implies  that  if  Cascade  had  been  run  with  search  control  turned 
off.  then  al>out  X9  ( 12%  of  the  740)  cases  where  analogical  search  control 
predicted  the  snl)jects'  rule  choice  would  now  become  ca.ses  where  its 
predictions  fail.  In  addition,  hand  analysis  of  the  19  cases  where  analog¬ 
ical  search  control  failed  to  inedict  the  subject's  choices  indicates  that 
mdy  2  cases  would  be  successfully  jrredicted  if  analogical  search  control 
were  turueci  off.  Thus,  if  Cascade  had  only  its  default  search  control,  it 
would  mispredict  lOb  cases  that  it  successfully  predicted  with  analogical 
search  control  turned  on.  so  its  accuracy  would  drop  to  G-'j.C'/t  as  oj)posed 
to  97.7%  with  analogical  search  control  enabled.  Thus,  analogical  search 
control  does  help. 

In  ord<'r  to  further  understand  why  analogical  search  control  failed 
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10  predict  It)  rule  choices,  each  was  analyzed.  .\11  were  generated  by  2 
sub  jects,  so  ( 'nscnde's  anah)gicnl  search  control  predicted  the  rule  choices 
for  7  of  the  subjects  with  100%  accuracy.  Moreover,  it  turns  out  that  in 

11  of  the  19  cases,  the  first  inference  made  by  the  subject  could  not  be 
niodehul  by  any  Cascade  rule.  Such  cases  indicate  an  inaccurate  model  of 
the  subject's  knowledge,  rather  than  an  inaccurate  model  of  the  subject's 
search  control. 

In  summary,  our  initial  result  appeared  to  suggest  that  analogical 
search  control  provided  no  closer  match  between  Cascade's  problem  solv¬ 
ing  behavior  and  the  subjects'  than  Cascade's  normal  problem  .solving 
dill.  However,  for  7  of  the  9  subjects.  Cascade  provides  a  clear  improve¬ 
ment  in  matching  the  subjects  when  analogical  search  control  is  used, 
lor  the  other  two  subjects,  the  failure  to  match  appears  to  arise  from 
missing  prior  knowledge  rather  than  a  defect  in  the  learning  mechanism. 
.Moreover,  if  analogical  search  control  is  turned  off.  Cascade's  prediction 
accuracy  would  drop. 

Discussion  of  the  fit  between  Cascade  and  individual  sub¬ 
jects 

There  were  two  |)urposes  in  fitting  Cascade  to  the  inrlividual  sub¬ 
jects  The  fi'st  was  lo  find  out  what  the  subjects  were  doing,  and  the 
second  was  to  find  out  how  well  Cascade  could  model  that.  We  discuss 
th<'se  objectives  together,  first  looking  at  e.xample-studying  behavior, 
then  p/oblem  sohing  behavior. 

The  two  major  processes  during  e.xample-studying  were  explanation 
of  a  line  and  acceptance  of  a  line  or  a  part  of  a  line  without  explaining  it. 
< 'as(  ado's  model  of  self-exi)lanation  is  to  rederive  the  line  via  ordinary 
(b'diK  lion.  Cascade's  i)ackwards  chaining  control  structure  ensures  that 
oid\  inferences  relevant  lo  the  top  goal  are  made.  This  sufficed  to  model 
of  the  snl>jects'  227  explantions  (see  Talde  (i).  Cascade's  model  of 
accept  ine  a  line  was  simply  to  inune  a  whole  subtree  of  an  explanation  In- 
accepting  a  goal  as  achieved  without  trying  to  achieve  it.  This  sufficed  lo 
account  for  92%  of  the  subjects'  422  cases  of  acceptance  (see  Table  -a); 
actually,  the  lack  of  fit  may  be  due  to  the  goal  structure  implicit  in 
Cascade  s  rules  rather  than  the  acceptance  mechanism  per  se.  Overall, 
(  ascade  accounts  for  about  7-5%  of  the  subjects'  behavior  during  exajnple 
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stud  yiiip,.** 

I'Ih-  s('ir-ox|)liinat ions  tiinf  Casrado  doos  not  ntodfl  arp  nioslly  (T'VA  ) 
foiKeiiK’d  willi  cognitive  skills  that  we  are  not  interested  in  modeling, 
such  as  algehiaic  e<(nation  solving.  That  left  only  23  explanations  that 
Cascade  really  sliould  have  modeled.  These  fell  into  two  groups;  in¬ 
come  1  explanations  (14  cases)  and  more  general  comments  (9  cases) 
in<  hiding  abstract,  partial  solution  plans  and  observations  such  as.  "The 
tensit)n  is  important  Irecause  it  transmits  force  between  the  blocks  of 
an  .Atwood's  machine."  The  first  group  indicates  that  Cascade  needs 
more  Imggy  rules  than  it  currently  has.  In  particular,  it  needs  to  model 
miscoiK epi ions  al>out  acceleration  and  motion.  The  second  group  indi¬ 
cated  lliat  the  subjects  have  an  ability  that  Cascade  lacks.  They  can 
stand  i>ack  from  the  details  and  alrstract  an  overall  view  of  either  the 
solution  (i.e..  they  see  an  abstract  plan  or  chronological  pattern  in  the 
inferences)  or  the  system  (i.e..  they  form  a  mental  model  of  the  me- 
(  hanica!  device).  .Although  these  are  certainly  interesting  and  important 
types  of  cognition,  they  appeared  surprisingly  rarely  in  this  study  (only 
of  227  cases,  or  4/1 ).  When  the  Cascade  research  began,  we  ex])ected 
plan  recognition  to  be  the  most  important  kind  of  self-explanation.  We 
have  since  learned  that  it  occurs  rarely  and  may  have  little  influence 
on  snbse(|nent  problem  solving.'*  Overall,  it  is  good  news  that  only  23 
(  1  r  i  )  of  the  Ititi  interesting,  task-domain  relevant  explanations  uttered 
by  subjects  recpiire  extensions  to  Cascade  in  order  to  model  them.  Even 
in  its  pies(»n(  form.  Cascade  successfully  models  the  bulk  of  the  subjects' 
self-explanal  ions. 

*4  lie  co\ciaKc  tinnic  for  example  sliulyiiig  wa.«  calculalerl  a.s  follows:  Table  (>  shows 
dial  m'4  of  die  subjects'  self-explanations  are  modeled  by  <’a.srade.  Table  S  shows 
dial  of  die  acceptances  are  moileled  by  Cascade.  However,  these  tables  use  dif¬ 
ferent  iiiiils  From  Table  we  can  esliiuale  that  about  40‘7  of  the  subjects'  behavior 
wa.s  aei  epI aiiees,  so  we  ( aii  use  that  fiRiire  to  form  a  weiRhted  average  of  the  two  cov¬ 
erages,  and  thus  I'ah  iilate  that  about  ~V/t  of  the  subjects'  example  stiulyiiig  behavior 
is  mall  lied  by  ( 'ascade. 

'  Mill  oiigli  il  seems  pointless  with  only  3  ca.ses  of  abstract  planning  in  the  example 
stiidiiiig  protocols,  we  could  analyze  the  problem  solving  protocols  to  see  If  these 
subjects'  rule  choices  during  the  relevant  sections  of  their  protocol  are  belter  explained 
by  the  abstract  plans  they  found  during  example  studying  than  by  the  existing  search 
control  mechanisms  of  Cascade  Because  analogical  search  control  probably  makes  the 
same  predictions  about  rule  choices  as  an  abstract  plan,  we  doubt  that  this  analysis 
would  vield  iiiieipiivocal  results. 
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Tilt*  two  most  coiiiiuoii  piol)lem  solving  processes  were  backwards 
(•liiiining  rnle-l);ise(l  inference  and  transformational  analogy.  W'e  were 
snrpiis«'d  l)\'  the  prevalence  of  transformational  analogy  during  problem 
solving,  alt  bough  it  was  certainly  due  in  part  to  the  fact  that  12  of  the  21 
problems  in  the  study  were  isomorjthic  (or  nearly  so)  to  one  of  the  three 
e.\ami)les  (i.e..  there  was  a  set  of  "string”  problems,  "incline”  problems, 
and  "pulley"  prolrlems).  .Although  only  around  b/f  of  subjects'  problem 
solving  involved  transformational  analogy  (see  Table  7).  it  often  had  a 
|)rofound  affect  on  the  direction  of  the  subjects'  search.  Cascade  has  a 
sim|)le  model  of  transformational  analogy,  but  it  wa.s  not  powerful  enough 
to  handle  all  the  cases.  Subjects  sometimes  find  analogical  mairpings 
that  ( 'asca<le  cannot.  They  sometimes  mi.x  transformational  and  regular 
problem  solving  (47  cases).  They  sometimes  prefer  to  use  transforma¬ 
tional  analogy  even  when  they  do  not  have  to  use  it  ca.ses).  Many  of 
these  problematic  ca.ses  occur  when  subjects  need  to  draw  a  free-body  di- 
agiam  and  refer  to  tlie  e.xainple's  free-body  diagram  for  hel|).  They  may 
be  using  well-honed  skills  for  visual  analogizing.  This  would  explain  why 
Cascade's  t  ransformat  iotial  analogy,  which  is  oriented  towards  analogical 
trans|(M'  of  e<|ualions.  is  so  incomplete. 

On  the  whole,  it  appears  that  most  of  the  exami)le-st udying  and 
problem-solving  behavior  can  be  explained  as  deduction,  simple  acce))- 
tance  of  example  statements,  atid  transfortnational  analogy.  .Al’hongh 
thesf'  three  pro(  esses  cover  oidy  T-Va  of  the  examirle  studying  behavior 
and  bO-ftO','  of  the  problem  solving  behavior,  the  behavior  they  do  not 
cover  mostly  involves  mathematical  manipulations  and  other  types  of 
cognition  that  are  outside  the  dotnain  of  sttidy. 

We  were  surprised  to  find  that  the  overall  control  structure  and  local 
control  choices  were  also  modeled  rather  accurately  by  Cascade  (see  the 
|)ie<  eding  section  ).  Hovs evei ,  we  did  not  stress  this  aspect  of  Cascade  ei- 
I  her  during  it  development  not  during  its  evaluation,  so  tiiere  is  jirobably 
mm  h  toom  for  improvement  in  both  areas. 

GENERAL  DISCUSSION 

VVe  have  completed  three  steps  of  a  four-step  research  program.  The 
first  sfpp  was  to  find  a  computationallv  sufficient  account  for  the  knowl¬ 
edge  a<  (position  that  ocrured  in  the  Chi  et  al.  study.  The  major  techni- 
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ral  linidlo  was  finding  a  way  to  constrain  search  during  problem  solving 
so  that  imi)assos  would  occur  at  the  right  jrlaces.  This  was  achieved 
In  atlding  analogical  search  control,  a  form  of  symbol-level  learning. 
There  was  no  way  to  tell  in  advance  of  running  Cascade  whether  ana- 
luRical  search  control  was  sufficient.  Fortunately,  it  was.  and  (.'ascade 
was  al>le  to  learn  all  the  rules  that  it  needed  to  learn.  Moreover,  the 
problem  of  getting  impasses  to  occur  in  the  right  places  is  faced  by  all 
impasse-driven  machine  learning  systems,  so  this  result  is  relevant  to 
many  machine  learning  systems.  A  minor  hurdle  was  finding  a  way  to 
transfer  knowledge  from  the  knot-is-a-body  example  line  to  problem  solv- 
imi,.  .\fter  trying  several  methods,  we  discovered  a  new  machine  learning 
le(  linitine,  which  we  called  analogical  abduction. 

riie  second  step  in  the  research  program  was  to  demonstrate  that 
(  'as(ade  could  explain  the  main  findings  of  the  Chi  et  al.  study.  .As  a 
model  of  the  self-ex|)lanation  effect.  Cascade  was  qualitatively  ad('(|nate. 
It  could  self-explain  example  lines  as  well  as  just  accept  them.  It  could 
solve  problems  with  and  without  referring  to  examples,  and  its  analogi¬ 
cal  references  can  both  dive  into  the  middle  of  the  example  to  i)ick  out 
a  single  fact  (analogical  search  control)  or  read  the  example  from  the 
becinning  searching  foi  a  useful  etptation  (transformational  analogy).  In 
order  to  go  beyond  (pialitative  similarity,  simulations  were  conducted 
that  modeled  ai\  idealized  good  student  and  an  idealized  poor  student. 
.\ll  four  of  the  main  findings  from  the  self-explanation  study  were  re- 
prod  uce<l  in  the  contrast  between  the  two  simulations.  .A  particularly 
sn|>rising  result  was  that  most  of  the  learning  occurred  during  problem 
solving  even  though  the  particular  learning  strategy  we  manipulated. 
self-explanatioi\.  oiierated  only  during  example  studying.  Examination 
of  (  'asc  ade's  processing  showed  that  the  acceleration  of  its  learning  dur¬ 
ing  |)rob|em  solving  was  caused  bv  (  1)  analogical  search  control  obtain¬ 
ing  more  guidance  from  the  exirerience  (derivation)  left  behind  by  self- 
explaining  the  examples,  and  (2)  junblem  solving  having  prereipiisite 
knowledge,  obtained  during  examirle  studying,  that  allowed  it  to  reach 
impasses  where  learning  could  approirriately  take  place. 

The  third  step  in  the  research  program  was  to  demonstate  that  Cas¬ 
cade  can  simulate  real  student  cognition  al  the  b  to  10  second  unit  of 
analvsis.  We  fit  Ca.scade  to  subject  protocols  by  forcing  it  to  explain 
>‘\a<  i|\  the  same  lines  as  the  subject  and  to  do  transformational  analogy 


ill  I'XiK  lly  lln‘  snino  |)oiiits  as  flie  subject.  \W  also  gave  it  initial  knowl¬ 
edge  tliat  approximated  tlie  snhject's  knowledge  jnst  prior  to  exi>laining 
llie  examples.  Selling  these  parameters,  plus  occasionally  “nudging"  the 
system,  siifliced  to  fit  Cascade  to  cover  most  of  the  subjects'  behavior. 
Sm  li  lit  I  ing  was  carried  out  for  all  9  .sul>jcct.s.  and  the  resulting  match  be- 
Iwi'eii  system  and  sultjecl  behavior  was  evaluated.  As  one  might  e.\])eci. 
given  that  Cascade  was  designed  to  model  the  subjects,  almost  every¬ 
thing  it  did  was  also  done  by  the  subjects.  Tables  .5  and  7  show  that 
over  9.')'/T  of  Cascade's  goals  occured  in  the  subjects'  behavior  and  were 
achieved  the  same  way  by  both  simulation  and  subject.  On  the  other 
hand,  when  the  sultjects'  behavior  is  analyzed  in  terms  of  goals  and  infer¬ 
ences.  about  7')’d  of  their  e.xample-studying  behavior  and  between  (iO'/I 
and  90' i  of  their  problem-solving  behavior  is  matched  by  Cascade  goals 
and  infeicnces.  Most  of  the  unmatcherl  behavior  concerns  mathematical 
m:nd|)nlat ions  and  other  skills  that  (.'ascade  was  not  intended  to  model. 
It  was  found  that  the  main  inaderpiacy  in  Cascade  is  its  simple  model  of 
1 1  anslormal ional  analogy.  Subjects  were  tpiite  clever  at  forming  useful 
analogies  with  the  examples,  and  especially  their  free-body  diagrams. 

1  he  fourth  step  in  the  research  program  is  to  use  the  fitted  models  of 
indiv  idual  subjects  to  find  out  imue  about  their  learning.  Unfoii unatelv. 
we  discovered  during  the  current  fitting  that  some  of  our  assumptions 
about  initial  knowledge  were  incorrect,  so  we  will  have  to  rerun  the 
lilting  exercise  with  new  assumi)lions  before  comlucting  these  analyses. 
Noiu'l  lieless.  a  few  speculative  remarks  can  be  made  on  the  basis  of  the 
exist  ing  analyses. 

\\'<'  were  sill  prised  that  there  were  so  few  clear-cut  cases  of  impasse- 
driven  learning  in  the  protocols.  Tables  5  and  7  show  that  the  subjects 
had  clear  signs  of  impasses  on  only  occassions  when  Cascade  did 
F.niC.  .-Mthongh  analyses  to  be  conducted  later  will  tell  us  exactly  why 
there  weie  so  few  clear  cases  of  learning,  it  ai)|)ears  that  it  is  ilue  to 
overuse  of  the  free-body  diagrams.  .Most  of  the  rules  learned  by  the  ide¬ 
alized  good  siudmit  simulation  are  used  during  the  initial  stage  of  solving 
a  probh'iii.  when  a  situation  is  analyzed  and  the  forces  and  accelerations 
are  found.  The  examples  cover  this  |)hase  by  merely  presenting  the  free- 
body  diagram  and  perhaps  adding  a  few  lines  of  explanation  for  any 
forces  that  they  consider  unobvious.  As  a  consequence,  most  subjects 
simpiv  accepted  the  free-body  diagrams  without  trying  to  explain  them. 


Jtiul  llius  missfd  tlie  oppoMuiiity  to  learn.  During  problem  solving,  the 
<lins,inms  wore  ap,ain  orornsed.  This  time  they  siibjerfs  tended  to  use 
transformational  analogy  to  adapt  an  e.xample's  free-body  diagram  in¬ 
stead  of  figuring  one  out  from  their  knowledge  of  physics.  Thus,  they 
would  miss  the  chance  to  do  impasse-driven  learning.  This  problem  was 
exacerl)at(>d  by  the  fact  that  most  of  the  25  problems  were  deliberately 
(onstnicted  so  as  to  have  free-body  diagrams  that  were  similar  to  ones 
in  the  e.xamples.  Subjects  who  used  transformational  analogy  for  these 
prol)lems  tended  to  get  them  right.  This  meant  that  siibiects  could  learn 
\ery  little  and  yet  still  get  high  scores.  For  instance,  one  subject  who  did 
not  know  about  normal  forces  nonetheless  got  all  of  the  "normal  force" 
problems  right.  In  short,  it  a|)pears  that  overuse  of  the  diagrams,  exac¬ 
erbated  by  the  design  of  the  study's  problem  set.  re<luces  the  number  of 
cases  of  impasse-driven  learning. 

On  the  other  hand,  it  could  also  be  that  subjects  had  instances  of 
impasse-driven  learning,  but  they  showed  no  overt  signs  of  them.  Most 
of  ilu'  cases  of  overt  imirasse-driven  learning  came  from  just  two  subjects 
w  ho  were  the  most  vocal  of  the  subjects.  It  is  likely  that  the  other  sub- 
i«'(  is  had  episodes  of  im|)asse-driven  learning  l)ut  did  not  re])ort  them. 
We  had  hoped  to  detect  these  silent  impasses  by  seeing  changes  in  the 
sub  jed's  Ix'havior.  That  is.  we  had  hoped  to  see  one  or  more  occasions 
wlu'ie  the  lo-be-leartied  rtile  could  be  api)lipd  but  was  not.  followed  by  a 
solid  siring  of  o(  casious  where  the  rule  was  applied.  The  learning  event 
would  be  somewhere  in  the  vicinity  of  the  transition  from  non-usage  to 
usage.  This  tvpe  of  analysis  succeeded  in  locating  silent  learning  events 
ill  lower  of  Hanoi  protocols  ( VanLehn.  lOOla)  and  finger  counting  pro¬ 
tocols  (Siegler  .V  .lenkins.  .Jones  ck.'  \'anLehn.  1991).  Unfortunately, 

most  subjects  in  this  study  either  always  used  a  rule  or  always  avoided 
it.  In  some  sense,  t  hey  had  to  do  t  his.  Sul)  jects  in  the  two  earlier  studies 
l<'arne<l  new  alternative  strategies  to  solve  a  problem  that  they  could 
aln-adv  solve.  Thus,  if  they  did  not  use  one  of  the  to-l)e-learned  rules, 
they  always  had  their  old  rule  to  use  instead.  This  was  not  the  case  in 
the  |)resent  study.  If  a  rule  was  missing  for  a  new  kind  of  force,  for  ex¬ 
ample.  then  the  subject's  only  alternative  to  learning  a  new  rule  was  to 
use  a  transformational  analogy.  Once  they  have  successfully  used  trans¬ 
formational  analogy  for  this  appearance  of  the  new  force,  they  would 
lend  to  us)‘  it  for  all  other  appearances.  In  this  fashion,  they  would  miss 
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tlic  (i))))ort unity  to  lenin  new  rules.  In  short,  if  a  subject  was  to  learn  a 
rule.  tli('\  tended  to  leain  it  on  the  first  orca.sion  that  it  was  possible  to 
leain  it.  If  they  userl  an  alternative  to  the  rule,  then  they  tended  to  kee]) 
using  that  alternative  through  the  end  of  the  study.  Thus,  we  found  few 
transitions  from  not  using  a  rule  to  using  a  rule,  and  we  have  little  solid 
evideme  for  silent  impasses. 

In  short,  it  appears  that  there  is  less  learning  in  the  protocols  than 
we  had  hoped,  although  this  might  be  partly  an  artifact  of  our  inability 
to  detect  silent  impasses.  Nonetheless,  there  is  much  to  be  learned  by 
examining  the  instances  of  learning  that  did  occur.  For  instance,  it  would 
!)('  good  to  find  out  if  analogical  search  control  really  ditl  play  a  role  in 
guiding  the  subjects  to  an  appropriate  impa.s.se. 

Cascade  is  based  on  the  assumption  that  the  self-explanation  effect  is 
due  solely  to  a  dilference  in  example-studying  habits  rather  than  a  dif- 
fenmce  in  irrior  knowledge.  Surprisingly,  this  assumption  held  up  even 
during  the  fitting  of  individual  protocols.  However,  we  exjrect  some 
(  hallenges  to  arise  during  the  next  set  of  analyses.  If  may  be  that  the 
sub  ji'cls'  policies  about  using  transformational  analogy  are  just  as  im¬ 
portant  as  self-explanation  in  determining  whether  learning  will  take 
]ila<('.  W'e  suspect  that  effective  learning  re<|uires  both  that  the  subject 
explain  an  example  and  that  they  try  not  to  refer  to  it  during  problem 
solv  ing  for  puriroses  of  obtaining  a  free-body  diagram  or  an  erpiation. 
On  the  other  hand,  referring  to  the  example  for  advice  on  which  rule  to 
( lioos('  (analogical  search  control)  should  be  encouraged.  Methodologi¬ 
cally.  the  comirlexity  of  tliis  speculative  prescription  shows  the  advantage 
of  simidat ion-based  analyses  of  behavior.  \  prescrijttion  based  on  just 
tlie  Chi  et .  al  study  would  be  sim|)ler  and  perhaps  not  as  effective. 

(  ascade  shows  promise  as  a  general  model  of  cognitive  skill  accpii- 
silion.  but  it  needs  considerable  work  beyond  fixing  its  model  of  trans- 
loiniational  analogy.  In  order  to  be  a  more  com|)lete  account  of  the 
plienomena  at  hand,  it  needs  a  model  of  analogical  retrieval  and  of  the 
<lifr<’ren(e  between  i)hysical  and  mental  references  to  the  exami)les.  We 
believe'  the  existing  mechanisms  can  also  handle  some  well-known  phe¬ 
nomena  of  skill  acquisition,  such  as  practice  and  transfer  effects,  but 
this  needs  to  be  demonstrated.  The  major  limitation  on  the  general¬ 
ity  of  Cascade  ;j  is  its  use  of  monotonic  reasoning.  With  the  help  of 
Ilolf  Ploetzner.  we  are  currently  incorporating  a  version  of  the  situation 


rniriiltis  wliicli  will  greatly  enhance  the  types  of  reasoning  Cascade  can 
modi'l.  and  tints  the  nnmher  of  task  domains  that  it  cati  model.  We 
are  enconrageil  to  e.xtend  ('ascade  to  become  a  more  complete,  more 
general  model  of  learning  by  its  similarity  to  other  theories  of  cognitive 
skill  ac(piisition  (e.g..  Anderson,  1990:  Schank.  1986).  It  is  considerably 
simph'i  than  those  theories  and  jtrobably  more  thoroughly  implemented 
and  tested.  We  hope  that  its  simplicity  and  empirical  adequacy  remain 
intat  t  as  it  is  e.xtended. 

Finally,  these  results  shed  some  light  on  the  possibility  of  using  ma- 
chities  to  acquire  knowledge  for  e.xpert  systems  from  ordinary  human 
insi  I  net  iunal  material.  .Any  A1  e.xpert  would  suspect  that  machines 
would  have  a  hard  time  learning  from  human  materials  because  they 
lack  common  sense.  It  turns  out  that  common  sense  was  important  for 
Cascade's  learning,  but  it  was  not  particularly  hard  to  provide  it.  Com¬ 
mon  sense  was  encoded  in  the  non-domain  knowledge  given  to  Cascade 
as  part  of  its  initial  knowledge  base.  Most  of  the  non-domain  knowl¬ 
edge  concert. ed  geometric  reasoning,  common-sense  phy.sical  reasoning 
about  pushes  and  imlls.  and  most  significantly,  overly  general  rules.  This 
know  ledge  was  used  during  EBLC  to  form  new  physics-s])ecific  domain 
kimw  ledge.  Hence,  common  sense  knowledge  was  crucial  because  it  heav- 
il\  constrained  learning.  On  the  other  hand,  it  was  not  particularly  hard 
to  figure  out  what  that  knowledge  should  be.  Whenever  Cascade  would 
reach  an  impasse  that  it  cottid  not  resolve  with  its  existing  common  sense 
knowledge,  it  was  usually  quite  simple  to  specify  that  knowledge.  .After 
all.  it  is  common  sense. 
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APPENDIX 

riiis  appendix  contains  an  example  of  a  snliject's  protocol  encoded 
at  the  level  oi  (  ascade-like  subgoals.  The  protocol  apjiears  in  the  left 
column,  and  the  encoding  appears  in  the  right.  An  “(R)"  in  the  left 
column  indicates  that  the  subject  is  reading  the  problem  aloud.  The 
(odine  procedure  consisted  of  pretending  that  the  subject  was  a  version 
nf  (  a^(  ade.  and  generating  the  problem-solving  trace  that  this  version 
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of  Cascade  would  have  to  generate  to  lead  to  the  utterances  found  in  the 
protficol.  in  ord<M  for  the  suhjert's  actions  to  fit  into  Cascade's  control 
structure,  it  was  soiiieliines  necessary  to  hypothesize  problem-solving 
goals  that  are  not  verl)alize<l. 

The  Cascade  model  contains  four  types  of  goals: 

•  value! X):  find  a  value  for  the  <|uantity  X. 

•  solve! X  =  Y):  find  a  solution  to  the  equation  X=V. 

•  retrieve! example):  find  an  example  similar  to  the  current  problem. 

•  etpiat ion!  .X ):  bud  an  ecpiation  that  can  be  used  to  compute  a  value 
for  X. 

r.a(  h  goal  a|)pears  with  an  “S;"  when  the  goal  is  posted  and  an  “F:" 
when  a  solution  to  the  goal  has  been  found.  In  addition,  the  model 
sometimes  must  explicitly  “backtrack"  over  some  subgoals  to  account 
for  backtracking  behavior  by  the  subject. 

The  coding  process  allowed  us  to  fit  the  hypothetical  Cascade  model 
as  closf'ly  as  possible  to  the  subject's  behavior.  In  doing  this,  we  were 
able  to  id«>ntify  s])erific  locations  at  which  the  current  implementation 
of  ('asrade  would  fail  to  generate  the  subject's  behavior.  At  these  loca¬ 
tions,  w('  would  have  to  manually  “midge"  the  system  onto  the  correct 
reasoning  path.  Events  of  this  type  are  marked  with  event  numbers  in 
till’  right -hatid  margin. 

This  particular  protocol  concerns  a  “poor"  subject  solving  a  combi¬ 
nation  pulley-incline  inolilem.  The  subject  initially  retrieves  the  pulley 
example  and  copies  the  eipiations  for  tension  and  acceleration  from  that 
example.  For  the  most  pai  t .  the  subject  attempts  to  directly  ap|)ly  these 
e(piat  ions  to  t  he  current  prolilem  ! a  strategy  that  will  leail  to  an  incorrect 
solution).  The  subject  computes  the  tension  of  the  string  and  the  accel¬ 
eration  of  the  free-hanging  block  in  this  manner.  However,  to  compute 
I  he  a(  (  eleral  ion  of  I  he  lilock  on  I  he  incline,  the  subject  assumes  his  result 
for  ac(  eleral  ion  from  the  cojiied  erpiations  is  actua’ly  the  projection  of  an 
acceleration  vector  that  points  down  the  incline.  The  subject  apparently 
does  not  have  a  rule  for  computing  the  projection  of  an  acceleration  vec¬ 
tor.  so  he  converts  it  to  a  force  vector  by  multiplying  the  acceleration 
l>\  the  mass  of  the  block  {F  =  wa).  He  then  computes  the  projection 
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of  I  lie  force  vector  and  converts  the  result  back  to  an  acceleration  with 
r  =  1)1(1  again. 
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problem:  q5 
Subject  S103 


Hypothetical  Cascade  model 


Okay . 

(R)  A  block  of  mass  ml  equals  3.0  slugs 
on  a  sss... smooth 

incline  plane  of  angle  30  degrees  is 
connected  by  a  cord 

over  a  small  frictionless  pulley  to  a 
second  block  of  mass 

m2  equals  2.0  slugs  hanging  vertically. 


(R)  What  is  the  acceleration  of  each 
body? 

Okay . 

That  would  just .  .  . 

Ahhh.  .  . 

Okay,  the  acceleration... 

That  would  be  M. . .with. . .that  would  be 
like  the  pulley 

again.  .  .this  equation,  (mvimbles) 

This  one. 


Okay . 

Urninm.  .  . 

Okay . 

Wait . 

If .  .  . 

Find  the  tension  of  the  whole... 

Okay,  I  want  to  find  the  tension  of  the 
whole  thing  again, 

and  then  I  can  find  the  acceleration 

for  each  one,  and  use 

that  to  find  force  for  ehh...for  the 

second. .. well  I'll  use 

it . 

Okay,  if  I  use  that  to  find  the  + 

component  of  the  force  like  | 

for  the  F . . .  j 

I 

The  Y  component  for  this  ml  block  on  | 
the  slant,  and  find  the  | 

whole  force,  then  I'll  find  the  whole  j 
acceleration  for  that.  j 

Okay.  j 

So  that  would  be  T  equals  2  times  M. . .  j 
Two  times  mass  one  is  3.0  times  mass  j 
two  is  2.0  over  mass  one  j 

plus  two,  is  3.0  plus  2.0... time  j 

slugs ...  I 


+ 

What  system  is  slugs?  + 
Where  is  that  table  again?  | 
Here .  | 
Slugs  is  feet  and  all.  j 
So  that  would  be. . , 


Soughts:  accel (ml) ,  accel (m2) , 
tension (cord) 

S:  value (accel (ml) ) ,  value (accel (m2) ) 


S :  retrieve (example) 

F:  retrieve (example) :  px 
S:  equation (ml) ,  equation(m2) 

S:  equation (ml) 

F:  equation (ml) :  T-mlg^mla 
S:  equation (m2) 

F:  equation (m2) :  T-m2^=m2a 
F:  equation (ml ) ,  equation (m2) : 
T-mlg=mla,  T-m2g=m2a 


S:  solve  (accel  (ml)  ■*  (1) 

Fn (pro j (accel (ml) ) ) ) , 
solve (accel (m2) * 
proj (accel (m2) ) ) 

S :  value (Fn (pro^ (accel (ml) ) ) ) ) , 
value (pro3 (accel (m2) ) ) 

S :  value (T)  [value (tension (cord) ) ] 


S:  equation (T) 

F:  equation(T):  ( (2mlm2) / (ml+m2)  )  g 
S:  solve (T= ( (2mlm2) / (ml+m2) ) g) 

S:  value (ml) 

F :  value (ml) : 3 . 0 
S:  value (m2) 

F :  value (m2) : 2 . 0 
S;  value (g) 

S:  value (system (problem) )  (2) 

S:  value (system (slugs) )  (3) 


F:  value (system (slugs) ) : 
british 

F:  value (system (problem) ) : 
british 


F:  value (g):  32 


Gravity  would  be  32  feet  per  second 
squared. 


Okay,  so  T  equals  2  times  3  is  6. 
Time... 12  (  )  is  36  times  2  is  12. 


12... time  32  equal ...  divided  by  5. 


Equals  76.8. 


Okay . 

And  the  acceleration. . . 

For  ml  the  accel . . . 

I'll  do  m2  first . 

Acceleration  for  m2  is  the  lighter 
one,  would  be . . . 

Ahh. . .T  minus  mlGm2V  equals  m2A. . . 
76.6  minus  2  times  32 


equals  2A. 

2A  equals  76.8  minus  64  would  be  12.8. 


A  would  be ...  6.4. 


And  for . . . 
ml .  .  . 

T  minus  mlG  ec[uals  negative  mlA 
Will  be  76.8  minus  3  times  32  equals 


S:  solve (T=( (2*3. 0*2.0) / 

(3. 0+2.0) ) *32) 

S:  value ( (2*3*2) / (3+2) ) *32) 

S:  value ( (2*3*2) *32) 

S:  value (2*3*2) 

S:  value (2*3) 

F :  value (2*3)  :  6 
S:  value (6*2) 

F:  value (6*2):  12 
F:  value (2*3*2) :  12 
S:  value (12*32) 

F:  value (12*32) :  384 
F:  value ( (2*3*2) *32) :  384 
S:  value (384/ (3+2) ) 

S:  value (3+2) 

F :  value  (3+2)  :  5 
S:  value (384/5) 

F:  value (384/5) :  76.8 
F:  value (384/ (3+2) ) :  76.8 
F:  value ( (2*3*2) / (3+2) ) *32) :  76.8 
F:  solve (T=( (2*3. 0*2.0)  / 

(3. 0+2.0) )*32) :  T=76.8 
F:  solve (T= ( (2mlm2) / (ml+m2) ) g)  : 

T-76.8 

F :  value (T) :  76.8 
S:  value (Fn (pro j (accel (ml) )) ) 
Backtrack  (4) 

S:  value (pro j (accel (m2) ) ) 


S:  solve (T-m2g»m2a) 

[a>=pro  j  (accel  (m2) )  ] 

S:  value (T) 

F :  value (T) :  76.8 
S:  value (m2) 

F:  value (m2):  2 
S;  value (g) 

F :  value (g) :  32 
S:  solve  (76. 8--2*32*2*a) 

S:  value (76. 8-2*32) 

S:  value (2*32) 

F:  value (2*32) :  64 
S:  value (76 . 8-64) 

F:  value (76. 8-64) :  12.8 
S;  value (76. 8-2*32) :  12.8 
S:  solve (12 . 8«2*a) 

S:  solve (a-12 . 8/2) 

S:  value (12 . 8/2) 

F:  value (12 . 8/2) :  6.4 
F:  solve (a*12 . 8/2) :  a*6.4 
F:  solve (12 . 8»2*a) :  a“6.4 
F:  solve (76. 8-2*32-2*a) :  a=6.4 
F:  solve (T-m2g*m2a) :  a=6 . 4 
F:  value (pro j (accel (m2) )) :  6.4 

S:  value (Fn (pro j (accel (ml) )) ) 

S:  value (pro j (accel (ml) ) ) 

S:  solve (T-mlg»-mla) 

[a«T3roj  (accel  (ml)  )  ] 
S:  value (T) 

F :  value (T) :  76.8 
S:  value (ml) 

F:  value (ml) :  3 
S:  value  (g) 


I 


negative  3A.  .  . 

That's  76.8  minus  (  )... 

76.8  minus  96  equals  negative  19.2. 


Negative  3A  equals  negative  19.2. 


So  A  equals  6.4  again. 


It's  meters  per... 

No  it  isn't. 

It's  feet  per  second  squared. 

This  is  feet  per  second  squared. 
Okay,  but  that's  just  the  vertical 
part . 

Cause  it's  not  a  slant. 

So  you  can  find. . . 

Force  equals  ma . . 


And  so  it  would  be  Fy  ecjuals  mAy 
Fy  equals  mass  is  3  slug  times  6.4 


equals  19.2  pounds. 


Okay,  then  F  would  equal  Fy  over  the 
sine  of  30  degrees, 
fif .  .  . 

Equal  19.2  over  the  sine  of  30 
degrees  equals  19.2  divided  by 
30  degrees  sine  equals  38.4  pounds. 


Okay,  then. 

Acceleration. . . 

F  equals  ma . . . 

38.4  pounds  equals .. .mass  is  3  slugs 
times  A. 


A  equals  12.8  feet  per  second  squared. 


F :  value (g) :  32 
S:  solve (76 . 8-3*32»-3*a) 

S:  value(76. 8-3*32) 

S:  value (3*32) 

F:  value(3*32):  96 
S:  value (76 . 8-96) 

F:  value (76 . 8-96) :  -19.2 
F:  value (76. 8-3*32)  :  -19.2 
S:  solve (-19 . 2*-3*a) 

S:  solve (a»-19 . 2/-3) 

S:  value (-19 . 2/-3) 

F:  value (-19 . 2/-3)  :  6.4 
F:  solve (a»-19 . 2/-3) :  a=6.4 
F:  solve (-19 . 2»-3*a) :  a“6 . 4 
F:  solve (76 . 8-3*32=-3*a) :  a=6 . 4 
F:  solve (T-mlg=-mla) :  a=6.4 
F:  value (pro j (accel (ml) )) :  6.4 


+  S:  solve (F»ma)  [a-accel (ml ) ]  (5) 

I  S:  solve (a=F/m) 

I  S:  value (F/m) 

I  S:  value (F) 

I  S:  solve (F=Fy/sin  30) 

1  S:  value (Fy/sin  30) 

1  S:  value (Fy) 

I  S:  solve (Fy*m*ay) 

I  [ay“proj (accel (ml) ) ] 

I  S:  value (m*ay) 

I  S:  value  (m) 

I  F :  value (m) :  3 

1  S:  value (ay) 

I  F :  value (ay) :  6.4 

I  S:  value (3*6.4) 

I  F:  value (3*6.4)  :  19.2 

1  F:  value (m*ay) :  19.2 

j  F:  solve (Fy=m*ay) :  F=19.2 

I  F:  value (Fy):  19.2 


S:  value (19 . 2/8in  30) 

F:  value (19 . 2/sin  30):  38.4 
F:  value(Fy/sin  30):  38.4 
F:  solve (F“Fy/sin  30):  F=38.4 
F :  value (F) :  38.4 


S:  value (m) 

F :  value (m) :  3 
S:  value (38 . 4/3) 

F:  value(38.4/3) :  12.8 
F:  value (F/m):  12.8 
F:  solve (a-F/m) :  a-12 . 8 
F :  solve (F-ma) :  a“12 . 8 
F:  value (Fn (pro j (accel (ml) ))) :  12.8 
F:  value (Fn (pro j (accel (ml) )))) , 


